Method and apparatus for processing video signal

ABSTRACT

An image decoding method according to the present invention may comprise the steps of: deriving a merge candidate from a candidate block; generating a first merge candidate list including the merge candidate; specifying any one of a plurality of merge candidates included in the first merge candidate list; deriving affine vectors of a current block on the basis of motion information of the specified merge candidate; deriving a motion vector of a sub-block in the current block on the basis of the affine vectors; and performing motion compensation on the sub-block on the basis of the motion vector.

TECHNICAL FIELD

The present invention relates to a method and an apparatus forprocessing video signal.

BACKGROUND ART

Recently, demands for high-resolution and high-quality images such ashigh definition (HD) images and ultra-high definition (UHD) images haveincreased in various application fields. However, higher resolution andquality image data has increasing amounts of data in comparison withconventional image data. Therefore, when transmitting image data byusing a medium such as conventional wired and wireless broadbandnetworks, or when storing image data by using a conventional storagemedium, costs of transmitting and storing increase. In order to solvethese problems occurring with an increase in resolution and quality ofimage data, high-efficiency image encoding/decoding techniques may beutilized.

Image compression technology includes various techniques, including: aninter-prediction technique of predicting a pixel value included in acurrent picture from a previous or subsequent picture of the currentpicture; an intra-prediction technique of predicting a pixel valueincluded in a current picture by using pixel information in the currentpicture; an entropy encoding technique of assigning a short code to avalue with a high appearance frequency and assigning a long code to avalue with a low appearance frequency; etc. Image data may beeffectively compressed by using such image compression technology, andmay be transmitted or stored.

In the meantime, with demands for high-resolution images, demands forstereographic image content, which is a new image service, have alsoincreased. A video compression technique for effectively providingstereographic image content with high resolution and ultra-highresolution is being discussed.

DISCLOSURE Technical Problem

The present invention is to provide a method and apparatus foreffectively performing inter-prediction on an encoding/decoding targetblock when encoding/decoding a video signal.

The present invention is to provide a method and apparatus forperforming motion compensation by using a plurality of merge candidatelists when encoding/decoding a video signal.

The present invention is to provide a method and apparatus forperforming motion compensation based on an affine model whendecoding/decoding a video signal.

The present invention is to provide a method and apparatus for derivinga corner motion vector from a neighboring block or a non-neighboringblock of a current block when decoding/decoding a video signal.

Technical problems obtainable from the present invention are non-limitedthe above-mentioned technical task, and other unmentioned technicaltasks can be clearly understood from the following description by thosehaving ordinary skill in the technical field to which the presentinvention pertains.

Technical Solution

A video signal decoding method and apparatus according to the presentinvention may derive a merge candidate from a candidate block, generatea first merge candidate list including the merge candidate, specify oneof a plurality of merge candidates included in the first merge candidatelist, derive affine vectors of a current block based on motioninformation of the specified merge candidate, derive a motion vector ofa sub-block in the current block based on the affine vectors and performmotion compensation on the sub-block based on the motion vector.

A video signal encoding method and apparatus according to the presentinvention may derive a merge candidate from a candidate block, generatea first merge candidate list including the merge candidate, specify oneof a plurality of merge candidates included in the first merge candidatelist, derive affine vectors of a current block based on motioninformation of the specified merge candidate, derive a motion vector ofa sub-block in the current block based on the affine vectors and performmotion compensation on the sub-block based on the motion vector.

For a video signal encoding/decoding method and apparatus according tothe present invention, when the candidate block is encoded by affineinter prediction, affine vectors of the merge candidate may be derivedbased on affine vectors of the candidate block.

For a video signal encoding/decoding method and apparatus according tothe present invention, positions of affine vectors of the candidateblock may be different based on whether the current block and thecandidate block are included in a same CTU (Coding Tree Unit).

For a video signal encoding/decoding method and apparatus according tothe present invention, affine vectors of the merge candidate may bederived by combining translation motion vectors of a plurality ofcandidate blocks.

For a video signal encoding/decoding method and apparatus according tothe present invention, when a number of merge candidates included in thefirst merge candidate list is less than a maximum number, a mergecandidate included in a second merge candidate list may be added to thefirst merge candidate list.

For a video signal encoding/decoding method and apparatus according tothe present invention, a number of affine parameters of the currentblock may be determined based on at least one of a size or shape of thecurrent block.

For a video signal encoding/decoding method and apparatus according tothe present invention, the candidate block may include at least one of aneighboring block of the current block and a non-neighboring block on asame line as the neighboring block.

It is to be understood that the foregoing summarized features areexemplary aspects of the following detailed description of the presentinvention without limiting the scope of the present invention.

Advantageous Effects

According to the present invention, efficiency of inter-prediction canbe enhanced by performing motion compensation by using a plurality ofmerge candidate lists.

According to the present invention, efficiency of inter-prediction canbe enhanced by obtaining motion information based on a plurality ofmerge candidates.

According to the present invention, efficiency of inter-prediction canbe enhanced by performing motion compensation based on an affine model.

According to the present invention, efficiency of inter-prediction canbe enhanced by deriving a corner motion vector from a neighboring blockor a non-neighboring block of a current block.

Effects obtainable from the present invention may be non-limited by theabove-mentioned effect, and other unmentioned effects can be clearlyunderstood from the following description by those having ordinary skillin the technical field to which the present invention pertains.

DESCRIPTION OF DRAWINGS

FIG. 1 is a block diagram illustrating a device for encoding a videoaccording to an embodiment of the present invention.

FIG. 2 is a block diagram illustrating a device for decoding a videoaccording to an embodiment of the present invention.

FIG. 3 is a diagram illustrating a partition mode candidate which may beapplied to a coding block when a coding block is encoded by interprediction.

FIG. 4 shows an example of hierarchically partitioning a coding blockbased on a tree structure as an embodiment to which the presentinvention is applied.

FIG. 5 is a diagram showing a partition shape in which a binarytree-based partitioning is allowed as an embodiment to which the presentinvention is applied.

FIG. 6 shows a triple tree partitioning shape.

FIG. 7 is a diagram showing an example in which only a specific shape ofbinary tree-based partitioning is allowed.

FIG. 8 is a diagram for describing an example in which informationrelated to a number of times allowed for a binary tree partitioning isencoded/decoded according to an embodiment to which the presentinvention is applied.

FIG. 9 is a flowchart illustrating an inter prediction method as anembodiment to which the present invention is applied.

FIG. 10 is a diagram illustrating a procedure of deriving motioninformation of a current block when a merge mode is applied to thecurrent block.

FIG. 11 is a diagram showing an example of a spatial neighboring block.

FIG. 12 is a diagram showing an example of deriving a motion vector of atemporal merge candidate.

FIG. 13 is a diagram showing a position of candidate blocks that arepossibly used as a co-located block.

FIG. 14 is a diagram showing a process of deriving motion information ofa current block when an AMVP mode is applied to the current block.

FIG. 15 is a diagram illustrating an example of deriving a mergecandidate from a second merge candidate block when a first mergecandidate block is unavailable.

FIG. 16 is a diagram illustrating an example of deriving a mergecandidate from a second merge candidate block positioned on the sameline as the first merge candidate block.

FIGS. 17 to 20 are diagrams illustrating the order of searching formerge candidate blocks.

FIG. 21 is a diagram illustrating an example in which a merge candidateof a non-square block is derived on the basis of a square block.

FIG. 22 is a diagram illustrating an example of deriving a mergecandidate on the basis of a high-level node block.

FIG. 23 is a diagram illustrating an example of determining availabilityof a spatial neighboring block on the basis of a merge estimationregion.

FIG. 24 is a diagram illustrating an example in which a merge candidateis derived on the basis of a merge estimation region.

FIG. 25 is a diagram illustrating a motion model.

FIG. 26 is a diagram illustrating an affine motion model using cornermotion vectors.

FIG. 27 is a diagram illustrating an example in which a motion vector isdetermined in a unit of a sub-block.

FIG. 28 is a diagram illustrating an example in which a position of acorner is determined according to a shape of a current block.

FIG. 29 is a flowchart illustrating a motion compensation process underan affine inter mode.

FIG. 30 and FIG. 31 are a diagram illustrating an example of deriving anaffine vector of a current block based on a motion vector of aneighboring block.

FIG. 32 is a diagram illustrating candidate blocks for deriving aspatial merge candidate.

FIG. 33 shows an example of deriving affine vectors of a current blockfrom affine vectors of a candidate block.

MODE FOR INVENTION

A variety of modifications may be made to the present invention andthere are various embodiments of the present invention, examples ofwhich will now be provided with reference to drawings and described indetail. However, the present invention is not limited thereto, and theexemplary embodiments can be construed as including all modifications,equivalents, or substitutes in a technical concept and a technical scopeof the present invention. The similar reference numerals refer to thesimilar element in described the drawings.

Terms used in the specification, ‘first’, ‘second’, etc. can be used todescribe various components, but the components are not to be construedas being limited to the terms. The terms are only used to differentiateone component from other components. For example, the ‘first’ componentmay be named the ‘second’ component without departing from the scope ofthe present invention, and the ‘second’ component may also be similarlynamed the ‘first’ component. The term ‘and/or’ includes a combination ofa plurality of items or any one of a plurality of terms.

In the present disclosure, when an element is referred to as being“connected” or “coupled” to another element, it is understood to includenot only that the element is directly connected or coupled to thatanother element but also that there may be another element therebetween.When an element is referred to as being “directly connected” or“directly coupled” to another element, it is understood that there is noother element therebetween.

The terms used in the present specification are merely used to describeparticular embodiments, and are not intended to limit the presentinvention. An expression used in the singular encompasses the expressionof the plural, unless it has a clearly different meaning in the context.In the present specification, it is to be understood that terms such as“including”, “having”, etc. are intended to indicate the existence ofthe features, numbers, steps, actions, elements, parts, or combinationsthereof disclosed in the specification, and are not intended to precludethe possibility that one or more other features, numbers, steps,actions, elements, parts, or combinations thereof may exist or may beadded.

Hereinafter, preferred embodiments of the present invention will bedescribed in detail with reference to the accompanying drawings.Hereinafter, the same constituent elements in the drawings are denotedby the same reference numerals, and a repeated description of the sameelements will be omitted.

FIG. 1 is a block diagram illustrating a device for encoding a videoaccording to an embodiment of the present invention.

Referring to FIG. 1, the device 100 for encoding a video may include: apicture partitioning module 110, prediction modules 120 and 125, atransform module 130, a quantization module 135, a rearrangement module160, an entropy encoding module 165, an inverse quantization module 140,an inverse transform module 145, a filter module 150, and a memory 155.

The constitutional parts shown in FIG. 1 are independently shown so asto represent characteristic functions different from each other in thedevice for encoding a video. Thus, it does not mean that eachconstitutional part is constituted in a constitutional unit of separatedhardware or software. In other words, each constitutional part includeseach of enumerated constitutional parts for convenience. Thus, at leasttwo constitutional parts of each constitutional part may be combined toform one constitutional part or one constitutional part may be dividedinto a plurality of constitutional parts to perform each function. Theembodiment where each constitutional part is combined and the embodimentwhere one constitutional part is divided are also included in the scopeof the present invention, if not departing from the essence of thepresent invention.

Also, some of constituents may not be indispensable constituentsperforming essential functions of the present invention but be selectiveconstituents improving only performance thereof. The present inventionmay be implemented by including only the indispensable constitutionalparts for implementing the essence of the present invention except theconstituents used in improving performance. The structure including onlythe indispensable constituents except the selective constituents used inimproving only performance is also included in the scope of the presentinvention.

The picture partitioning module 110 may partition an input picture intoone or more processing units. Here, the processing unit may be aprediction unit (PU), a transform unit (TU), or a coding unit (CU). Thepicture partitioning module 110 may partition one picture intocombinations of multiple coding units, prediction units, and transformunits, and may encode a picture by selecting one combination of codingunits, prediction units, and transform units with a predeterminedcriterion (e.g., cost function).

For example, one picture may be partitioned into multiple coding units.A recursive tree structure, such as a quad tree structure, may be usedto partition a picture into coding units. A coding unit which ispartitioned into other coding units with one picture or a largest codingunit as a root may be partitioned with child nodes corresponding to thenumber of partitioned coding units. A coding unit which is no longerpartitioned by a predetermined limitation serves as a leaf node. Thatis, when it is assumed that only square partitioning is possible for onecoding unit, one coding unit may be partitioned into four other codingunits at most.

Hereinafter, in the embodiment of the present invention, the coding unitmay mean a unit performing encoding, or a unit performing decoding.

A prediction unit may be one of partitions partitioned into a square ora rectangular shape having the same size in a single coding unit, or aprediction unit may be one of partitions partitioned so as to have adifferent shape/size in a single coding unit.

When a prediction unit subjected to intra prediction is generated basedon a coding unit and the coding unit is not the smallest coding unit,intra prediction may be performed without partitioning the coding unitinto multiple prediction units N×N.

The prediction modules 120 and 125 may include an inter predictionmodule 120 performing inter prediction and an intra prediction module125 performing intra prediction. Whether to perform inter prediction orintra prediction for the prediction unit may be determined, and detailedinformation (e.g., an intra prediction mode, a motion vector, areference picture, etc.) according to each prediction method may bedetermined. Here, the processing unit subjected to prediction may bedifferent from the processing unit for which the prediction method anddetailed content is determined. For example, the prediction method, theprediction mode, etc. may be determined by the prediction unit, andprediction may be performed by the transform unit. A residual value(residual block) between the generated prediction block and an originalblock may be input to the transform module 130. Also, prediction modeinformation, motion vector information, etc. used for prediction may beencoded with the residual value by the entropy encoding module 165 andmay be transmitted to a device for decoding a video. When a particularencoding mode is used, it is possible to transmit to a device fordecoding video by encoding the original block as it is withoutgenerating the prediction block through the prediction modules 120 and125.

The inter prediction module 120 may predict the prediction unit based oninformation of at least one of a previous picture or a subsequentpicture of the current picture, or may predict the prediction unit basedon information of some encoded regions in the current picture, in somecases. The inter prediction module 120 may include a reference pictureinterpolation module, a motion prediction module, and a motioncompensation module.

The reference picture interpolation module may receive reference pictureinformation from the memory 155 and may generate pixel information of aninteger pixel or less then the integer pixel from the reference picture.In the case of luma pixels, an 8-tap DCT-based interpolation filterhaving different filter coefficients may be used to generate pixelinformation of an integer pixel or less than an integer pixel in unitsof a ¼ pixel. In the case of chroma signals, a 4-tap DCT-basedinterpolation filter having different filter coefficient may be used togenerate pixel information of an integer pixel or less than an integerpixel in units of a ⅛ pixel.

The motion prediction module may perform motion prediction based on thereference picture interpolated by the reference picture interpolationmodule. As methods for calculating a motion vector, various methods,such as a full search-based block matching algorithm (FBMA), a threestep search (TSS), a new three-step search algorithm (NTS), etc., may beused. The motion vector may have a motion vector value in units of a ½pixel or a ¼ pixel based on an interpolated pixel. The motion predictionmodule may predict a current prediction unit by changing the motionprediction method. As motion prediction methods, various methods, suchas a skip method, a merge method, an AMVP (Advanced Motion VectorPrediction) method, an intra block copy method, etc., may be used.

The intra prediction module 125 may generate a prediction unit based onreference pixel information neighboring to a current block which ispixel information in the current picture. When the neighboring block ofthe current prediction unit is a block subjected to inter prediction andthus a reference pixel is a pixel subjected to inter prediction, thereference pixel included in the block subjected to inter prediction maybe replaced with reference pixel information of a neighboring blocksubjected to intra prediction. That is, when a reference pixel is notavailable, at least one reference pixel of available reference pixelsmay be used instead of unavailable reference pixel information.

Prediction modes in intra prediction may include a directionalprediction mode using reference pixel information depending on aprediction direction and a non-directional prediction mode not usingdirectional information in performing prediction. A mode for predictingluma information may be different from a mode for predicting chromainformation, and in order to predict the chroma information, intraprediction mode information used to predict luma information orpredicted luma signal information may be utilized.

In performing intra prediction, when the size of the prediction unit isthe same as the size of the transform unit, intra prediction may beperformed on the prediction unit based on pixels positioned at the left,the top left, and the top of the prediction unit. However, in performingintra prediction, when the size of the prediction unit is different fromthe size of the transform unit, intra prediction may be performed usinga reference pixel based on the transform unit. Also, intra predictionusing N×N partitioning may be used for only the smallest coding unit.

In the intra prediction method, a prediction block may be generatedafter applying an AIS (Adaptive Intra Smoothing) filter to a referencepixel depending on the prediction modes. The type of the AIS filterapplied to the reference pixel may vary. In order to perform the intraprediction method, an intra prediction mode of the current predictionunit may be predicted from the intra prediction mode of the predictionunit neighboring to the current prediction unit. In prediction of theprediction mode of the current prediction unit by using mode informationpredicted from the neighboring prediction unit, when the intraprediction mode of the current prediction unit is the same as the intraprediction mode of the neighboring prediction unit, informationindicating that the prediction modes of the current prediction unit andthe neighboring prediction unit are equal to each other may betransmitted using predetermined flag information. When the predictionmode of the current prediction unit is different from the predictionmode of the neighboring prediction unit, entropy encoding may beperformed to encode prediction mode information of the current block.

Also, a residual block including information on a residual value whichis a different between the prediction unit subjected to prediction andthe original block of the prediction unit may be generated based onprediction units generated by the prediction modules 120 and 125. Thegenerated residual block may be input to the transform module 130.

The transform module 130 may transform the residual block including theinformation on the residual value between the original block and theprediction unit generated by the prediction modules 120 and 125 by usinga transform method, such as discrete cosine transform (DCT), discretesine transform (DST), and KLT. Whether to apply DCT, DST, or KLT inorder to transform the residual block may be determined based on intraprediction mode information of the prediction unit used to generate theresidual block.

The quantization module 135 may quantize values transformed to afrequency domain by the transform module 130. Quantization coefficientsmay vary depending on the block or importance of a picture. The valuescalculated by the quantization module 135 may be provided to the inversequantization module 140 and the rearrangement module 160.

The rearrangement module 160 may rearrange coefficients of quantizedresidual values.

The rearrangement module 160 may change a coefficient in the form of atwo-dimensional block into a coefficient in the form of aone-dimensional vector through a coefficient scanning method. Forexample, the rearrangement module 160 may scan from a DC coefficient toa coefficient in a high frequency domain using a zigzag scanning methodso as to change the coefficients to be in the form of one-dimensionalvectors. Depending on the size of the transform unit and the intraprediction mode, vertical direction scanning where coefficients in theform of two-dimensional blocks are scanned in the column direction orhorizontal direction scanning where coefficients in the form oftwo-dimensional blocks are scanned in the row direction may be usedinstead of zigzag scanning. That is, which scanning method among zigzagscanning, vertical direction scanning, and horizontal direction scanningis used may be determined depending on the size of the transform unitand the intra prediction mode.

The entropy encoding module 165 may perform entropy encoding based onthe values calculated by the rearrangement module 160. Entropy encodingmay use various encoding methods, for example, exponential Golombcoding, context-adaptive variable length coding (CAVLC), andcontext-adaptive binary arithmetic coding (CABAC).

The entropy encoding module 165 may encode a variety of information,such as residual value coefficient information and block typeinformation of the coding unit, prediction mode information, partitionunit information, prediction unit information, transform unitinformation, motion vector information, reference frame information,block interpolation information, filtering information, etc. from therearrangement module 160 and the prediction modules 120 and 125.

The entropy encoding module 165 may entropy encode the coefficients ofthe coding unit input from the rearrangement module 160.

The inverse quantization module 140 may inversely quantize the valuesquantized by the quantization module 135 and the inverse transformmodule 145 may inversely transform the values transformed by thetransform module 130. The residual value generated by the inversequantization module 140 and the inverse transform module 145 may becombined with the prediction unit predicted by a motion estimationmodule, a motion compensation module, and the intra prediction module ofthe prediction modules 120 and 125 such that a reconstructed block canbe generated.

The filter module 150 may include at least one of a deblocking filter,an offset correction unit, and an adaptive loop filter (ALF).

The deblocking filter may remove block distortion that occurs due toboundaries between the blocks in the reconstructed picture. In order todetermine whether to perform deblocking, the pixels included in severalrows or columns in the block may be a basis of determining whether toapply the deblocking filter to the current block. When the deblockingfilter is applied to the block, a strong filter or a weak filter may beapplied depending on required deblocking filtering strength. Also, inapplying the deblocking filter, horizontal direction filtering andvertical direction filtering may be processed in parallel.

The offset correction module may correct offset with the originalpicture in units of a pixel in the picture subjected to deblocking. Inorder to perform the offset correction on a particular picture, it ispossible to use a method of applying offset in consideration of edgeinformation of each pixel or a method of partitioning pixels of apicture into the predetermined number of regions, determining a regionto be subjected to perform offset, and applying the offset to thedetermined region.

Adaptive loop filtering (ALF) may be performed based on the valueobtained by comparing the filtered reconstructed picture and theoriginal picture. The pixels included in the picture may be divided intopredetermined groups, a filter to be applied to each of the groups maybe determined, and filtering may be individually performed for eachgroup. Information on whether to apply ALF and a luma signal may betransmitted by coding units (CU). The shape and filter coefficient of afilter for ALF may vary depending on each block. Also, the filter forALF in the same shape (fixed shape) may be applied regardless ofcharacteristics of the application target block.

The memory 155 may store the reconstructed block or picture calculatedthrough the filter module 150. The stored reconstructed block or picturemay be provided to the prediction modules 120 and 125 in performinginter prediction.

FIG. 2 is a block diagram illustrating a device for decoding a videoaccording to an embodiment of the present invention.

Referring to FIG. 2, the device 200 for decoding a video may include: anentropy decoding module 210, a rearrangement module 215, an inversequantization module 220, an inverse transform module 225, predictionmodules 230 and 235, a filter module 240, and a memory 245.

When a video bitstream is input from the device for encoding a video,the input bitstream may be decoded according to an inverse process ofthe device for encoding a video.

The entropy decoding module 210 may perform entropy decoding accordingto an inverse process of entropy encoding by the entropy encoding moduleof the device for encoding a video. For example, corresponding to themethods performed by the device for encoding a video, various methods,such as exponential Golomb coding, context-adaptive variable lengthcoding (CAVLC), and context-adaptive binary arithmetic coding (CABAC)may be applied.

The entropy decoding module 210 may decode information on intraprediction and inter prediction performed by the device for encoding avideo.

The rearrangement module 215 may perform rearrangement on the bitstreamentropy decoded by the entropy decoding module 210 based on therearrangement method used in the device for encoding a video. Therearrangement module may reconstruct and rearrange the coefficients inthe form of one-dimensional vectors to the coefficient in the form oftwo-dimensional blocks. The rearrangement module 215 may receiveinformation related to coefficient scanning performed in the device forencoding a video and may perform rearrangement via a method of inverselyscanning the coefficients based on the scanning order performed in thedevice for encoding a video.

The inverse quantization module 220 may perform inverse quantizationbased on a quantization parameter received from the device for encodinga video and the rearranged coefficients of the block.

The inverse transform module 225 may perform the inverse transform,i.e., inverse DCT, inverse DST, and inverse KLT, which is the inverseprocess of transform, i.e., DCT, DST, and KLT, performed by thetransform module on the quantization result by the device for encoding avideo. Inverse transform may be performed based on a transfer unitdetermined by the device for encoding a video. The inverse transformmodule 225 of the device for decoding a video may selectively performtransform schemes (e.g., DCT, DST, and KLT) depending on multiple piecesof information, such as the prediction method, the size of the currentblock, the prediction direction, etc.

The prediction modules 230 and 235 may generate a prediction block basedon information on prediction block generation received from the entropydecoding module 210 and previously decoded block or picture informationreceived from the memory 245.

As described above, like the operation of the device for encoding avideo, in performing intra prediction, when the size of the predictionunit is the same as the size of the transform unit, intra prediction maybe performed on the prediction unit based on the pixels positioned atthe left, the top left, and the top of the prediction unit. Inperforming intra prediction, when the size of the prediction unit isdifferent from the size of the transform unit, intra prediction may beperformed using a reference pixel based on the transform unit. Also,intra prediction using N×N partitioning may be used for only thesmallest coding unit.

The prediction modules 230 and 235 may include a prediction unitdetermination module, an inter prediction module, and an intraprediction module. The prediction unit determination module may receivea variety of information, such as prediction unit information,prediction mode information of an intra prediction method, informationon motion prediction of an inter prediction method, etc. from theentropy decoding module 210, may divide a current coding unit intoprediction units, and may determine whether inter prediction or intraprediction is performed on the prediction unit. By using informationrequired in inter prediction of the current prediction unit receivedfrom the device for encoding a video, the inter prediction module 230may perform inter prediction on the current prediction unit based oninformation of at least one of a previous picture or a subsequentpicture of the current picture including the current prediction unit.Alternatively, inter prediction may be performed based on information ofsome pre-reconstructed regions in the current picture including thecurrent prediction unit.

In order to perform inter prediction, it may be determined for thecoding unit which of a skip mode, a merge mode, an AMVP mode, and aninter block copy mode is used as the motion prediction method of theprediction unit included in the coding unit.

The intra prediction module 235 may generate a prediction block based onpixel information in the current picture. When the prediction unit is aprediction unit subjected to intra prediction, intra prediction may beperformed based on intra prediction mode information of the predictionunit received from the device for encoding a video. The intra predictionmodule 235 may include an adaptive intra smoothing (AIS) filter, areference pixel interpolation module, and a DC filter. The AIS filterperforms filtering on the reference pixel of the current block, andwhether to apply the filter may be determined depending on theprediction mode of the current prediction unit. AIS filtering may beperformed on the reference pixel of the current block by using theprediction mode of the prediction unit and AIS filter informationreceived from the device for encoding a video. When the prediction modeof the current block is a mode where AIS filtering is not performed, theAIS filter may not be applied.

When the prediction mode of the prediction unit is a prediction mode inwhich intra prediction is performed based on the pixel value obtained byinterpolating the reference pixel, the reference pixel interpolationmodule may interpolate the reference pixel to generate the referencepixel of an integer pixel or less than an integer pixel. When theprediction mode of the current prediction unit is a prediction mode inwhich a prediction block is generated without interpolation thereference pixel, the reference pixel may not be interpolated. The DCfilter may generate a prediction block through filtering when theprediction mode of the current block is a DC mode.

The reconstructed block or picture may be provided to the filter module240. The filter module 240 may include the deblocking filter, the offsetcorrection module, and the ALF.

Information on whether or not the deblocking filter is applied to thecorresponding block or picture and information on which of a strongfilter and a weak filter is applied when the deblocking filter isapplied may be received from the device for encoding a video. Thedeblocking filter of the device for decoding a video may receiveinformation on the deblocking filter from the device for encoding avideo, and may perform deblocking filtering on the corresponding block.

The offset correction module may perform offset correction on thereconstructed picture based on the type of offset correction and offsetvalue information applied to a picture in performing encoding.

The ALF may be applied to the coding unit based on information onwhether to apply the ALF, ALF coefficient information, etc. receivedfrom the device for encoding a video. The ALF information may beprovided as being included in a particular parameter set.

The memory 245 may store the reconstructed picture or block for use as areference picture or block, and may provide the reconstructed picture toan output module.

As described above, in the embodiment of the present invention, forconvenience of explanation, the coding unit is used as a termrepresenting a unit for encoding, but the coding unit may serve as aunit performing decoding as well as encoding.

In addition, a current block may represent a target block to beencoded/decoded. And, the current block may represent a coding treeblock (or a coding tree unit), a coding block (or a coding unit), atransform block (or a transform unit), a prediction block (or aprediction unit), or the like depending on an encoding/decoding step. Inthis description, ‘unit’ may represent a base unit for performing aspecific encoding/decoding process and ‘block’ may represent apredetermined-sized sample array. Unless otherwise specified, ‘block’and ‘unit’ may be used as the same meaning. For example, in theafter-mentioned example, it may be understood that a coding block and acoding unit mutually have the same meaning.

One picture may be encoded/decoded by being divided into base blockshaving a square shape or a non-square shape. At this time, the baseblock may be referred to as a coding tree unit. The coding tree unit maybe defined as a coding unit of the largest size allowed within asequence or a slice. Information representing whether the coding treeunit has a square shape or has a non-square shape or informationregarding a size of the coding tree unit may be signaled through asequence parameter set, a picture parameter set, or a slice header. Thecoding tree unit may be divided into smaller size partitions. At thistime, if it is assumed that a depth of a partition generated by dividingthe coding tree unit is 1, a depth of a partition generated by dividingthe partition having depth 1 may be defined as 2. That is, a partitiongenerated by dividing a partition having a depth k in the coding treeunit may be defined as having a depth k+1.

A partition of arbitrary size generated by dividing a coding tree unitmay be defined as a coding unit. The coding unit may be recursivelydivided or divided into base units for performing prediction,quantization, transform, or in-loop filtering, and the like. Forexample, a partition of arbitrary size generated by dividing the codingunit may be defined as a coding unit, or may be defined as a transformunit or a prediction unit, which is a base unit for performingprediction, quantization, transform or in-loop filtering and the like.

Alternatively, a prediction block having the same size as a coding blockor smaller than a coding block may be determined by the predictionpartitioning of a coding block. For the prediction partitioning of acoding block, any one of partition mode (Part_mode) candidatesrepresenting a partitioning shape of a coding block may be specified.Information for determining a partition index indicating any one ofpartition mode candidates may be signaled through a bitstream.Alternatively, the partition index of a coding block may be determinedbased on at least one of a size, a shape, or an encoding mode of acoding block. The size or shape of a prediction block may be determinedbased on a partition mode specified by the partition index. A partitionmode candidates may include an asymmetric partition shape (e.g., nL×2N,nR×2N, 2N×nU, 2N×nD). The number or type of asymmetric partition modecandidates available for a coding block may be determined based on atleast one of the size, a shape, or an encoding mode of a coding block.

FIG. 3 is a diagram illustrating a partition mode candidate which may beapplied to a coding block when a coding block is encoded by interprediction.

When a coding block is encoded by inter prediction, any one of 8partition mode candidates shown in FIG. 3 may be applied to a codingblock.

On the other hand, when a coding block is encoded by intra prediction,only a square partition partitioning may be applied to a coding block.In other words, when a coding block is encoded by intra prediction, apartition mode, PART_2N×2N or PART_N×N, may be applied to a codingblock.

PART_N×N may be applied when a coding block has the minimum size.Herein, the minimum size of a coding block may be predefined in anencoder and a decoder. Alternatively, information regarding the minimumsize of a coding block may be signaled through a bitstream. In anexample, the minimum size of a coding block may be signaled through aslice header. Accordingly, the minimum size of a coding block may bedifferently determined per slice.

In another example, a partition mode candidate available for a codingblock may be differently determined according to at least one of thesize or shape of a coding block. In an example, the number or type ofpartition mode candidates available for a coding block may bedifferently determined according to at least one of the size or shape ofa coding block.

Alternatively, the type or number of asymmetric partition modecandidates available for a coding block may be determined based on thesize or shape of a coding block. The number or type of asymmetricpartition mode candidates available for a coding block may bedifferently determined according to at least one of the size or shape ofa coding block. In an example, when a coding block has a non-squareshape that width is greater than height, at least one of PART_2N×N,PART_2N×nU or PART_2N×nD may not be used as a partition mode candidateof a coding block. When a coding block has a non-square shape thatheight is greater than width, at least one of PART_N×2N, PART_nL×2N,PART_nR×2N may not be used as a partition mode candidate of a codingblock.

Generally, a prediction block may have a 4×4 to 64×64 size. But, when acoding block is encoded by inter prediction, a prediction block may berestricted not to have a 4×4 size to reduce memory bandwidth inperforming motion compensation.

Based on a partition mode, a coding block may be recursivelypartitioned. In other words, based on a partition mode determined by apartition index, a coding block may be partitioned and each partitiongenerated by partitioning a coding block may be defined as a codingblock.

Hereinafter, a method of partitioning a coding unit will be described inmore detail. In the after-mentioned example, a coding unit may mean acoding tree unit or a coding unit included in a coding tree unit. Inaddition, ‘a partition’ generated by partition a coding block may mean‘a coding block.’ The after-mentioned partitioning method may be appliedwhen a coding block is partitioned into a plurality of prediction blocksor transform blocks.

A coding unit may be partitioned by at least one line. In this case, anangle of a line which partitions a coding unit may be a value within arange of 0 to 360 degree. For example, the angle of a horizontal linemay be 0 degree, the angle of a vertical line may be 90 degree, theangle of a diagonal line in a right-top direction may be 45 degree andthe angle of a left-top diagonal line may be 135 degree.

When a coding unit is partitioned by a plurality of lines, all of aplurality of lines may have the same angle. Alternatively, at least oneof a plurality of lines may have a different angle from other lines.Alternatively, a plurality of lines partitioning a coding tree unit or acoding unit may have a predefined angle difference (e.g., 90 degree).

Information regarding a line partitioning a coding unit may bedetermined by a partition mode. Alternatively, information on at leastone of the number, direction, angle or position in a block of a line maybe encoded.

For convenience of description, in the after-mentioned example, it isassumed that a coding unit is partitioned into a plurality of codingunits by using at least one of a vertical line or a horizontal line.

The number of vertical lines or horizontal lines partitioning a codingunit may be at least one or more. In an example, a coding unit may bepartitioned into 2 partitions by using one vertical line or onehorizontal line. Alternatively, a coding unit may be partitioned into 3partitions by using two vertical lines or two horizontal lines.Alternatively, a coding unit may be partitioned into 4 partitions ofwhich width and height are half of a coding unit, by using one verticalline or one horizontal line.

When a coding unit is partitioned into a plurality of partitions byusing at least one vertical line or at least one horizontal line,partitions may have a uniform size. Alternatively, one partition mayhave a different size from other partitions or each partition may have adifferent size. In an example, when a coding unit is partitioned by twohorizontal lines or two vertical lines, a coding unit may be partitionedinto 3 partitions. In this case, a width or height ratio of 3 partitionsmay be n:2n:n, 2n:n:n, or n:n:2n.

In the after-mentioned examples, partitioning a coding block into 4partitions is referred to as a quad tree-based partitioning. And,partitioning a coding block into 2 partitions is referred to as a binarytree-based partitioning. In addition, partitioning a coding block into 3partitions is referred to as a triple tree-based partitioning.

In the after-mentioned drawing, it will be shown that one vertical lineand/or one horizontal line is used to partition a coding unit, but itwill be described that partitioning a coding unit into more partitionsthan shown by using more vertical lines and/or more horizontal linesthan shown or partitioning a coding unit into fewer partitions thanshown is also included in the range of the present invention.

FIG. 4 shows an example of hierarchically partitioning a coding blockbased on a tree structure as an embodiment to which the presentinvention is applied.

An input video signal is decoded in a predetermined block unit and abase unit for decoding an input video signal is referred to as a codingblock. A coding block may be a unit of performing intra/interprediction, transform and quantization. In addition, a prediction mode(e.g., an intra prediction mode or an inter prediction mode) may bedetermined in a unit of a coding block and prediction blocks included ina coding block may share a determined prediction mode. A coding blockmay be a square or non-square block in an arbitrary size within a rangeof 8×8 to 64×64 or a square or non-square block with a size of 128×128,256×256 or more.

Specifically, a coding block may be hierarchically partitioned based onat least one of a quad tree partitioning method, a binary tree partitionmethod or a triple tree partitioning method. A quad tree-basedpartitioning may mean a method in which a 2N×2N coding block ispartitioned into four N×N coding blocks. A binary tree-basedpartitioning may mean a method in which one coding block is partitionedinto two coding blocks. A triple tree-based partitioning may mean amethod in which one coding block is partitioned into three codingblocks. Even when triple tree or binary tree-based partitioning isperformed, a square coding block may exist at a lower depth.

Partitions generated by binary tree-based partitioning may be symmetricor asymmetric. In addition, a coding block partitioned based on a binarytree may be a square block or a non-square block (e.g., a rectangle).

FIG. 5 is a diagram showing a partitioning shape for a coding blockbased on binary tree partitioning. A partition shape of a coding blockbased on binary tree partitioning may include a symmetric type such as2N×N (a non-square coding unit in a horizontal direction) or N×2N (anon-square coding unit in a vertical direction), etc. or an asymmetrictype such as nL×2N, nR×2N, 2N×nU or 2N×nD, etc. Only one of thesymmetric type or the asymmetric type may be allowed as a partitioningshape of a coding block.

A triple tree partitioning shape may include at least one of a shapepartitioning a coding block into 2 vertical lines or a shapepartitioning a coding block into 2 horizontal lines. 3 non-squarepartitions may be generated by triple tree partitioning.

FIG. 6 shows a triple tree partitioning shape.

A triple tree partitioning shape may include a shape partitioning acoding block into 2 horizontal lines or a shape partitioning a codingblock into 2 vertical lines. The width or height ratio of partitionsgenerated by partitioning a coding block may be n:2n:n, 2n:n:n orn:n:2n.

The position of a partition with the largest width or height among 3partitions may be predefined in an encoder and a decoder. Alternatively,information indicating a partition with the largest width or heightamong 3 partitions may be signaled through a bitstream.

Only partitioning of a square shape or a non-square symmetric shape maybe allowed for a coding unit. In this case, partitioning a coding unitinto square partitions may correspond to quad tree CU partitioning, andpartitioning a coding unit into non-square partitions in a symmetricshape may correspond to binary tree partitioning. Partitioning a codingtree unit into square partitions and non-square partitions in asymmetric shape may correspond to quad tree and binary tree CUpartitioning (QTBT).

Binary tree or triple tree-based partitioning may be performed for acoding block in which quad tree-based partitioning is not performed anymore. A coding block generated by binary tree or triple tree-basedpartitioning may be partitioned into smaller coding blocks. In thiscase, at least one of quad tree partitioning, triple tree partitioningor binary tree partitioning may be set not to be applied to the codingblock. Alternatively, a binary tree partitioning in a predetermineddirection or a triple tree partitioning in a predetermined direction maynot be allowed for the coding block. In an example, quad treepartitioning and triple tree partitioning may be set to be unallowablefor a coding block generated by binary tree or triple tree-basedpartitioning. Only binary tree partitioning may be allowed for thecoding block.

Alternatively, only the largest coding block among 3 coding blocksgenerated by triple tree-based partitioning may be partitioned intosmaller coding blocks. Alternatively, binary tree-based partitioning ortriple tree-based partitioning may be allowed only for the largestcoding block among 3 coding blocks generated by triple tree-basedpartitioning.

The partitioning shape of a lower depth partition may be dependentlydetermined based on the partitioning shape of an upper depth partition.In an example, when an upper partition and a lower partition arepartitioned based on a binary tree, only binary tree-based partitioningin the same shape as a binary tree partitioning shape of an upper depthpartition may be allowed for a lower depth partition. For example, whenthe binary tree partitioning shape of an upper depth partition is 2N×N,the binary tree partitioning shape of a lower depth partition may bealso set to be 2N×N. Alternatively, when the binary tree partitioningshape of an upper depth partition is N×2N, the partitioning shape of alower depth partition may be also set to be N×2N.

Alternatively, binary tree partitioning in the same partitioningdirection as an upper depth partition or triple tree partitioning in thesame partitioning direction as an upper depth partition may be set to beunallowable for the largest partition among partitions generated bytriple tree-based partitioning.

Alternatively, the partitioning shape of a lower depth partition may bedetermined by considering the partitioning shape of an upper depthpartition and the partitioning shape of a neighboring lower depthpartition. Concretely, if an upper depth partition is partitioned basedon a binary tree, the partitioning shape of a lower depth partition maybe determined so that the same result as that of partitioning an upperdepth partition based on a quad tree does not occur. In an example, whenthe partitioning shape of an upper depth partition is 2N×N and thepartitioning shape of a neighboring lower depth partition is N×2N, thepartitioning shape of a current lower depth partition may not be set tobe N×2N. It is because when the partitioning shape of a current lowerdepth partition is N×2N, it causes the same result as that ofpartitioning an upper depth partition based on a N×N-shaped quad tree.When the partitioning shape of an upper depth partition is N×2N and thepartitioning shape of a neighboring lower depth partition is 2N×N, thepartitioning shape of a current lower depth partition may not be set tobe 2N×N. In other words, when the binary tree partitioning shape of anupper depth partition is different from the binary tree partitioningshape of a neighboring lower depth partition, the binary treepartitioning shape of a current lower depth partition may be set thesame as the binary tree partitioning shape of an upper depth partition.

Alternatively, the binary tree partitioning shape of a lower depthpartition may be set to be different from the binary tree partitioningshape of an upper depth partition.

An allowable binary tree partitioning shape may be determined in a unitof a sequence, a slice or a coding unit. In an example, a binary treepartitioning shape allowable for a coding tree unit may be limited to a2N×N or N×2N shape. An allowable partitioning shape may be predefined inan encoder or a decoder. Alternatively, information on an allowablepartitioning shape or an unallowable partitioning shape may be encodedand signaled through a bitstream.

FIG. 7 is a diagram showing an example in which only a specific shape ofbinary tree-based partitioning is allowed.

FIG. 7 (a) represents an example in which only N×2N-shaped binarytree-based partitioning is allowed and FIG. 7 (b) represents an examplein which only 2N×N-shaped binary tree-based partitioning is allowed.

To represent various partitioning shapes, information on quad treepartitioning, information on binary tree partitioning or information ontriple tree partitioning may be used. Information on quad treepartitioning may include at least one of information indicating whetherquad tree-based partitioning is performed or information on thesize/depth of a coding block in which quad tree-based partitioning isallowed. Information on binary tree partitioning may include at leastone of information indicating whether binary tree-based partitioning isperformed, information on whether binary tree-based partitioning is avertical direction or a horizontal direction, information on thesize/depth of a coding block in which binary tree-based partitioning isallowed or information on the size/depth of a coding block in whichbinary tree-based partitioning is not allowed. Information on tripletree partitioning may include at least one of information indicatingwhether triple tree-based partitioning is performed, information onwhether triple tree-based partitioning is a vertical direction or ahorizontal direction, information on the size/depth of a coding block inwhich triple tree-based partitioning is allowed or information on thesize/depth of a coding block in which triple tree-based partitioning isnot allowed. Information on the size of a coding block may represent atleast one minimum value or maximum value among the width, height,product of width and height or ratio of width and height of a codingblock.

In an example, when the width or height of a coding block is smallerthan the minimum size in which binary tree partitioning is allowed, orwhen the partitioning depth of a coding block is greater than themaximum depth in which binary tree partitioning is allowed, binarytree-based partitioning may not be allowed for the coding block.

In an example, when the width or height of a coding block is smallerthan the minimum size in which triple tree partitioning is allowed, orwhen the partitioning depth of a coding block is greater than themaximum depth in which triple tree partitioning is allowed, tripletree-based partitioning may not be allowed for the coding block.

Information on a condition that binary tree or triple tree-basedpartitioning is allowed may be signaled through a bitstream. Theinformation may be encoded in a unit of a sequence, a picture or apartial image. The partial image may mean at least one of a slice, atile group, a tile, a brick, a coding block, a prediction block or atransform block.

In an example, a syntax, ‘max_mtt_depth_idx_minus1’, representing themaximum depth that binary tree/triple tree partitioning is allowed maybe encoded/decoded through a bitstream. In this case,max_mtt_depth_idx_minus1+1 may indicate the maximum depth that binarytree/triple tree partitioning is allowed.

In an example, at least one of the number of times that binarytree/triple tree partitioning is allowed, the maximum depth that binarytree/triple tree partitioning is allowed or the number of depths thatbinary tree/triple tree partitioning is allowed may be signaled in asequence or a slice level. Accordingly, at least one of the number oftimes that binary tree/triple tree partitioning is allowed, the maximumdepth that binary tree/triple tree partitioning is allowed or the numberof depths that binary tree/triple tree partitioning is allowed may bedifferent for a first slice and a second slice. In an example, while forthe first slice, binary tree/triple tree partitioning may be allowedonly in one depth, for the second slice, binary tree/triple treepartitioning may be allowed in two depths.

In an example shown in FIG. 8, FIG. 8 shows that binary treepartitioning is performed for a coding unit having a depth of 2 and acoding unit having a depth of 3. Accordingly, at least one ofinformation representing the number of times (2 times) that binary treepartitioning is performed in a coding tree unit, informationrepresenting the maximum depth (depth 3) of a partition generated bybinary tree partitioning in a coding tree unit or informationrepresenting the number of partition depths (2 depths, depth 2 and depth3) that binary tree partitioning is applied in a coding tree unit may beencoded/decoded through a bitstream.

Alternatively, the number of times that binary tree/triple treepartitioning is allowed, a depth that binary tree/triple treepartitioning is allowed or the number of depths that binary tree/tripletree partitioning is allowed may be predefined in an encoder and adecoder. Alternatively, the number of times that binary tree/triple treepartitioning is allowed, a depth that binary tree/triple treepartitioning is allowed or the number of depths that binary tree/tripletree partitioning is allowed may be determined based on at least one ofan index of a sequence or a slice or the size/shape of a coding unit. Inan example, for a first slice, binary tree/triple tree partitioning maybe allowed in one depth and for a second slice, binary tree/triple treepartitioning may be allowed in two depths.

In another example, at least one of the number of times that binary treepartitioning is allowed, a depth that binary tree partitioning isallowed or the number of depths that binary tree partitioning is allowedmay be set differently according to a temporal level identifier(TemporalID) of a slice or a picture. Herein, the temporal levelidentifier (TemporalID) is for identifying each of a plurality of layersin an image having at least one or more scalabilities of view, spatial,temporal or quality.

As shown in FIG. 4, the first coding block 300 with the partitioningdepth (split depth) of k may be partitioned into multiple second codingblocks based on a quad tree. For example, the second coding blocks 310to 340 may be a square block having the half width and height of thefirst coding block and the partitioning depth of the second coding blockmay be increased to k+1.

The second coding block 310 with the partitioning depth of k+1 may bepartitioned into multiple third coding blocks with the partitioningdepth of k+2. Partitioning of the second coding block 310 may beperformed by selectively using one of a quad tree or a binary treedepending on a partitioning method. In this case, the partitioningmethod may be determined based on at least one of information indicatingquad tree-based partitioning or information indicating binary tree-basedpartitioning.

When the second coding block 310 is partitioned based on a quad tree,the second coding block 310 may be partitioned into four third codingblocks 310 a having the half width and height of the second coding blockand the partitioning depth of the third coding block 310 a may beincreased to k+2. On the other hand, when the second coding block 310 ispartitioned based on a binary tree, the second coding block 310 may bepartitioned into two third coding blocks. In this case, each of twothird coding blocks may be a non-square block having one of the halfwidth and height of the second coding block and the partitioning depthmay be increased to k+2. The second coding block may be determined as anon-square block in a horizontal direction or a vertical directionaccording to a partitioning direction and the partitioning direction maybe determined based on information on whether binary tree-basedpartitioning is performed in a vertical direction or a horizontaldirection.

Meanwhile, the second coding block 310 may be determined as a leafcoding block that is no longer partitioned based on a quad tree or abinary tree and in this case, the corresponding coding block may be usedas a prediction block or a transform block.

Like partitioning of the second coding block 310, the third coding block310 a may be determined as a leaf coding block or may be furtherpartitioned based on a quad tree or a binary tree.

On the other hand, the third coding block 310 b partitioned based on abinary tree may be further partitioned into coding blocks 310 b-2 in avertical direction or coding blocks 310 b-3 in a horizontal directionbased on a binary tree and the partitioning depth of the correspondingcoding block may be increased to k+3. Alternatively, the third codingblock 310 b may be determined as a leaf coding block 310 b-1 that is nolonger partitioned based on a binary tree and in this case, thecorresponding coding block 310 b-1 may be used as a prediction block ora transform block. However, the above-mentioned partitioning process maybe limitedly performed based on at least one of information on thesize/depth of a coding block that quad tree-based partitioning isallowed, information on the size/depth of a coding block that binarytree-based partitioning is allowed or information on the size/depth of acoding block that binary tree-based partitioning is not allowed.

The number of candidates that represent a size of a coding block may belimited to a predetermined number or a size of a coding block in apredetermined unit may have a fixed value. In an example, the size of acoding block in a sequence or in a picture may be limited to having anyof 256×256, 128×128 or 32×32. Information representing the size of acoding block in a sequence or in a picture may be signaled in a sequenceheader or a picture header.

As a result of partitioning based on a quad tree and a binary tree, acoding unit may be represented as a square or rectangular shape in anarbitrary size.

As shown in FIG. 4, the first coding block 300 with the partitioningdepth (split depth) of k may be partitioned into multiple second codingblocks based on a quad tree. For example, the second coding blocks 310to 340 may be a square block having the half width and height of thefirst coding block and the partitioning depth of the second coding blockmay be increased to k+1.

The second coding block 310 with the partitioning depth of k+1 may bepartitioned into multiple third coding blocks with the partitioningdepth of k+2. Partitioning of the second coding block 310 may beperformed by selectively using one of a quad tree or a binary treedepending on a partitioning method. In this case, the partitioningmethod may be determined based on at least one of information indicatingquad tree-based partitioning or information indicating binary tree-basedpartitioning.

When the second coding block 310 is partitioned based on a quad tree,the second coding block 310 may be partitioned into four third codingblocks 310 a having the half width and height of the second coding blockand the partitioning depth of the third coding block 310 a may beincreased to k+2. On the other hand, when the second coding block 310 ispartitioned based on a binary tree, the second coding block 310 may bepartitioned into two third coding blocks. In this case, each of twothird coding blocks may be a non-square block having one of the halfwidth and height of the second coding block and the partitioning depthmay be increased to k+2. The second coding block may be determined as anon-square block in a horizontal direction or a vertical directionaccording to a partitioning direction and the partitioning direction maybe determined based on information on whether binary tree-basedpartitioning is performed in a vertical direction or a horizontaldirection.

Meanwhile, the second coding block 310 may be determined as a leafcoding block that is no longer partitioned based on a quad tree or abinary tree and in this case, the corresponding coding block may be usedas a prediction block or a transform block.

Like partitioning of the second coding block 310, the third coding block310 a may be determined as a leaf coding block or may be furtherpartitioned based on a quad tree or a binary tree.

On the other hand, the third coding block 310 b partitioned based on abinary tree may be further partitioned into coding blocks 310 b-2 in avertical direction or coding blocks 310 b-3 in a horizontal directionbased on a binary tree and the partitioning depth of the correspondingcoding block may be increased to k+3. Alternatively, the third codingblock 310 b may be determined as a leaf coding block 310 b-1 that is nolonger partitioned based on a binary tree and in this case, thecorresponding coding block 310 b-1 may be used as a prediction block ora transform block. However, the above-mentioned partitioning process maybe limitedly performed based on at least one of information on thesize/depth of a coding block that quad tree-based partitioning isallowed, information on the size/depth of a coding block that binarytree-based partitioning is allowed or information on the size/depth of acoding block that binary tree-based partitioning is not allowed.

The number of candidates that represent a size of a coding block may belimited to a predetermined number or a size of a coding block in apredetermined unit may have a fixed value. In an example, the size of acoding block in a sequence or in a picture may be limited to having anyof 256×256, 128×128 or 32×32. Information representing the size of acoding block in a sequence or in a picture may be signaled in a sequenceheader or a picture header.

As a result of partitioning based on a quad tree and a binary tree, acoding unit may be represented as a square or rectangular shape in anarbitrary size.

A transform skip may be set not to be used for a coding unit generatedby binary tree-based partitioning or triple tree-based partitioning.Alternatively, a transform skip may be set to be applied to at least oneof a vertical direction or a horizontal direction in a non-square codingunit. In an example, when a transform skip is applied to a horizontaldirection, it represents only scaling is performed in a horizontaldirection without transform/inverse transform and transform/inversetransform using DCT or DST is performed in a vertical direction. When atransform skip is applied to a vertical direction, it represents onlyscaling is performed in a vertical direction without transform/inversetransform and transform/inverse transform using DCT or DST is performedin a horizontal direction.

Information on whether inverse transform for a horizontal direction isskipped or information on whether inverse transform for a verticaldirection is skipped may be signaled through a bitstream. In an example,information on whether inverse transform for a horizontal direction isskipped may be a 1-bit flag, ‘hor_transform_skip_flag’, and informationon whether inverse transform for a vertical direction is skipped may bea 1-bit flag, ‘ver_transform_skip_flag’.

An encoder may determine whether ‘hor_transform_skip_flag’ or‘ver_transform_skip_flag’ is encoded according to the size and/or shapeof a current block. In an example, when a current block has a N×2Nshape, hor_transform_skip_flag may be encoded and the encoding ofver_transform_skip_flag may be omitted. When a current block has a 2N×Nshape, ver_transform_skip_flag may be encoded andhor_transform_skip_flag may be omitted.

Alternatively, based on the size and/or shape of a current block,whether a transform skip for a horizontal direction is performed orwhether a transform skip for a vertical direction is performed may bedetermined. In an example, when a current block has a N×2N shape, atransform skip may be applied to a horizontal direction andtransform/inverse transform may be performed for a vertical direction.When a current block has a 2N×N shape, a transform skip may be appliedto a vertical direction and transform/inverse transform may be performedfor a horizontal direction. Transform/inverse transform may be performedbased on at least one of DCT or DST.

As a result of partitioning based on a quad tree, a binary tree or atriple tree, a coding block which is not partitioned any more may beused as a prediction block or a transform block. In other words, acoding block generated by quad tree partitioning or binary treepartitioning may be used as a prediction block or a transform block. Inan example, a prediction image may be generated in a unit of a codingblock and a residual signal, a difference between an original image anda prediction image, may be transformed in a unit of a coding block. Togenerate a prediction image in a unit of a coding block, motioninformation may be determined based on a coding block or an intraprediction mode may be determined based on a coding block. Accordingly,a coding block may be encoded by using at least one of a skip mode,intra prediction or inter prediction.

Alternatively, a plurality of coding blocks generated by partitioning acoding block may be set to share at least one of motion information, amerge candidate, a reference sample, a reference sample line or an intraprediction mode. In an example, when a coding block is partitioned by atriple tree, partitions generated by partitioning the coding block mayshare at least one of motion information, a merge candidate, a referencesample, a reference sample line or an intra prediction mode according tothe size or shape of a coding block. Alternatively, only part of aplurality of coding blocks may be set to share the information andremaining coding blocks may be set not to share the information.

In another example, it is possible to use a prediction block or atransform block smaller than a coding block by partitioning the codingblock.

Hereinafter, a method of performing inter prediction for a coding blockor a prediction block generated by partitioning the coding block will bedescribed in detail.

FIG. 9 is a flowchart illustrating an inter prediction method as anembodiment to which the present invention is applied.

Referring to FIG. 9, motion information of a current block may bedetermined S910. The motion information of the current block may includeat least one of a motion vector of the current block, a referencepicture index of the current block, an inter prediction direction or aweight of weighted prediction of the current block. The weight ofweighted prediction may represent a weight applying to an L0 referenceblock and a weight applying to an L1 reference block.

The motion vector of the current block may be determined on the basis ofinformation signaled through a bitstream. The precision of the motionvector represents the basic unit for expressing the motion vector of thecurrent block. For example, the precision of motion vector of thecurrent block may be determined to be one of an integer pel, a ½ pel, a¼ pel, or a ⅛ pel. The precision of motion vector may be determined on aper-picture basis, a per-slice basis, a per-tile group basis, a per-tilebasis, or a per-block basis. The block may represent a coding tree unit,a coding unit, a prediction unit, or a transform unit.

The motion information of the current block may be obtained based on atleast one of information signaled through a bitstream or motioninformation of a neighboring block neighboring the current block.

FIG. 10 is a diagram illustrating a procedure of deriving motioninformation of a current block when a merge mode is applied to thecurrent block.

A merge mode represents a method of deriving motion information of acurrent block from a neighboring block.

When a merge mode is applied to a current block, a spatial mergecandidate may be derived from a spatial neighboring block of a currentblock S1010. The spatial neighboring block may include at least one of ablock adjacent to a top boundary, left boundary, or corner (e.g., atleast one of a top left corner, a right top corner, or a left bottomcorner) of the current block.

FIG. 11 is a diagram showing an example of a spatial neighboring block.

As an example shown in FIG. 11, a spatial neighboring block may includeat least one of a neighboring block A₁ adjacent to a left of a currentblock, a neighboring block B1 adjacent to a top of the current block, aneighboring block A₀ adjacent to a bottom-left corner of the currentblock, a neighboring block B₀ adjacent to a top-right corner of thecurrent block, and a neighboring block B₂ adjacent to a top-left cornerof the current block. For example, let's assumed that a position of atop left corner sample of the current block is (0, 0), a width of thecurrent block is W, and a height of the current block is H. The block A₁may include a sample at position (−1, H−1). The block B₁ may include asample at position (W−1, −1). The block A₀ may include a sample atposition (−1, H). The block B₀ may include a sample at position (W, −1).The block B₂ may include a sample at position (−1, −1).

Expanding further an example of FIG. 11, a spatial merge candidate maybe derived from a block adjacent to a top-left sample of a currentblock, or a block adjacent to a top-center sample of the current block.For example, the block neighboring to the top-left sample of the currentblock may include at least of a block including a sample at position (0,−1) or a block including a sample at position (−1, 0). Or, a spatialmere candidate may be derived from at least one of a block neighboringto a top-center sample of the current block or a block neighboring to aleft-center sample of the current block. For example, the blockneighboring to the top-center sample of the current block may include asample at position (W/2, −1). The block neighboring to the left-centersample of the current block may include a sample at position (−1, H/2).

Based on the size and/or shape of a current block, the position of a topneighboring block and/or left neighboring block used to derive a spatialmerge candidate may be determined. In an example, when the size of acurrent block is greater than a threshold value, spatial mergecandidates may be derived from a block neighboring to the top centralsample of a current block and a block neighboring to the left centralsample of a current block. On the other hand, when the size of a currentblock is smaller than the threshold value, spatial merge candidates maybe derived from a block neighboring to the top-right sample of a currentblock and a block neighboring to the bottom-left sample of a currentblock. Herein, the size of a current block may be expressed based on atleast one of width, height, sum of width and height, product of widthand height or a ratio of width and height. A threshold value may be aninteger such as 2, 4, 8, 16, 32 or 128.

According to a shape of a current block, availability of an expandedspatial neighboring block may be determined. In an example, when acurrent block is a non-square block where a width is greater than aheight, it may be determined that a block adjacent to a top-left sampleof the current block, a block adjacent to a left-center sample, or ablock adjacent to a bottom-left sample of the current block is notavailable. Meanwhile, when a current block is a block where a height isgreater than a width, it may be determined that a block adjacent to atop-left sample of the current block, a block adjacent to a top-centersample, or a block adjacent to a top-right sample of the current blockis not available.

Motion information of a spatial merge candidate may be set to beidentical to motion information of a spatial neighboring block.

A spatial merge candidate may be determined by searching of neighboringblocks in a predetermined order. In an example, in an example shown inFIG. 11, searching for determining a spatial merge candidate may beperformed in an order of blocks A₁, B₁, B₀, A₀, and B₂. Herein, a blockB₂ may be used when at least one of remaining blocks (that is, A₁, B₁,B₀, and A₀) is not present or at least one is encoded through anintra-prediction mode.

An order of searching for a spatial merge candidate may be predefined inthe encoder/decoder. Alternatively, an order of searching for a spatialmerge candidate may be adaptively determined according to a size orshape of a current block. Alternatively, an order of searching for aspatial merge candidate may be determined on the basis of informationsignaled through a bitstream.

A temporal merge candidate may be derived from a temporal neighboringblock of a current block S1020. The temporal neighboring block may meana co-located block included in a co-located picture. The co-locatedpicture has a POC differing from a current picture including the currentblock. The co-located picture may be determined as a picture having apredefined index within a reference picture list or as a picture havinga POC difference with the current picture being minimum. Alternatively,the co-located picture may be determined by information signaled througha bitstream. Information signaled through a bitstream may include atleast one of information indicating a reference picture list (e.g., L0reference picture list or L1 reference picture list) including theco-located picture and an index indicating the co-located picture withinthe reference picture list. Information for determining the co-locatedpicture may be signaled in at least one of a picture parameter set, aslice header, and a block level.

Motion information on a temporal merge candidate may be determined onthe basis of motion information a co-located block. In an example, amotion vector of a temporal merge candidate may be determined on thebasis of a motion vector of a co-located block. For example, a motionvector of a temporal merge candidate may be set to be identical to amotion vector of a co-located block. Alternatively, a motion vector of atemporal merge candidate may be derived by scaling a motion vector of aco-located block on the basis of at least one of a POC differencebetween a current picture and a reference picture of the current block,and a POC difference between a co-located picture and a referencepicture of the co-located.

FIG. 12 is a diagram showing an example of deriving a motion vector of atemporal merge candidate.

In an example shown in FIG. 12, tb represents a POC difference between acurrent picture curr_pic and a reference picture curr_ref of the currentpicture, and td represents a POC difference between a co-located picturecol_pic and a reference picture col ref of the co-located block. Amotion vector of a temporal merge candidate may be derived by scaling amotion vector of the co-located block col_PU on the basis of tb and/ortd.

Alternatively, taking into account of whether or not a co-located blockis usable, a motion vector of the co-located block and a motion vectorobtained by scaling the motion vector of the co-located block may beused as a motion vector of a temporal merge candidate. In an example, amotion vector of a co-located block is set as a motion vector of a firsttemporal merge candidate, and a value obtained by scaling the motionvector of the co-located block may be set as a motion vector of a secondtemporal merge candidate.

An inter-prediction direction of a temporal merge candidate may be setto be identical to an inter-prediction direction of a temporalneighboring block. However, a reference picture index of the temporalmerge candidate may have a fixed value. In an example, a referencepicture index of a temporal merge candidate may be set to “0”.Alternatively, a reference picture index of a temporal merge candidatemay be adaptively determined on the basis of at least one of a referencepicture index of a spatial merge candidate, a reference picture index ofa current picture.

A specific block having the same position and size with a current blockwithin a co-located picture, or a block adjacent to a block adjacent toa block having the same position and size with the current block may bedetermined as a co-located block.

FIG. 13 is a diagram showing a position of candidate blocks that arepossibly used as a co-located block.

A candidate block may include at least one of a block adjacent to aposition of a top-left corner of a current block within a co-locatedpicture, a block adjacent to a position of a center sample of thecurrent block within the co-located picture, and a block adjacent to aposition of a bottom-left corner of the current block within theco-located picture.

In an example, a candidate block may include at least one of a block TLincluding a position of a top-left sample of a current block within aco-located picture, a block BR including a position of a bottom-rightsample of the current block within the co-located picture, a block Hadjacent to a bottom-right corner of the current block within theco-located picture, a block C3 including a position of a center sampleof the current block within the co-located picture, and a block C0adjacent to the center sample of the current block (for example, a blockincluding a position of a sample spaced apart from the center sample ofthe current block by (−1, −1)) within the co-located picture.

In addition to the example shown in FIG. 13, a block including aposition of a neighboring block adjacent to a predetermined boundary ofa current block within the co-located picture may be selected as aco-located block.

The number of temporal merge candidates may be 1 or more. In an example,at least one temporal merge candidate may be derived on the basis of atleast one co-located block.

Information on the maximum number of temporal merge candidates may beencoded and signaled through the encoder. Alternatively, the maximumnumber of temporal merge candidates may be derived on the basis of themaximum number of merge candidates and/or the maximum number of spatialmerge candidates which are possible included in a merge candidate list.Alternatively, the maximum number of temporal merge candidates may bedetermined on the basis of the number of usable co-located blocks.

Whether or not candidate blocks are usable may be determined accordingto a predetermined priority, and at least one co-located block may bedetermined on the basis of the above determination and the maximumnumber of temporal merge candidates. In an example, when a block C3including a position of a center sample of a current block and a block Hadjacent to a bottom-right corner of the current block are candidateblocks, any one of the block C3 and the block H may be determined as aco-located block. When the block H is available, the block H may bedetermined as a co-located block. However, when the block H is notavailable (for example, when the block H is encoded throughintra-prediction, when the block H is not usable or when the block H ispositioned outside of the largest coding unit (LCU), etc.), a block C3may be determined as a co-located block.

In another example, when at least one of a plurality of blocks adjacentto a bottom-right corner position of a current block within a co-locatedpicture is unavailable (for example, a block H and/or a block BR), theunavailable block may be replaced with another available block. Anotheravailable block that is replaced with a unavailable block may include atleast one a block (for example, C0 and/or C3) adjacent to a centersample position of a current block within a co-located picture, and ablock (for example, TL) adjacent to a bottom-left corner of the currentblock with the co-located picture.

When at least one of a plurality of blocks adjacent to a center sampleposition of a current block within a co-located picture is unavailableor when at least one of a plurality of blocks adjacent to a top-leftcorner position of the current block within the co-located picture isunavailable, the unavailable block may be replaced with anotheravailable block.

Subsequently, a merge candidate list including the spatial mergecandidate and the temporal merge candidate may be generated S1030. Whenconfiguring a merge candidate list, a merge candidate having motioninformation identical with an existing merge candidate may be removedfrom the merge candidate list.

Information on the maximum number of merge candidates may be signaledthrough a bitstream. In an example, information indicating the maximumnumber of merge candidates may be signaled through a sequence parameteror picture parameter. In an example, when the maximum number of mergecandidates is six, a total of six may be selected from spatial mergecandidates and temporal merge candidates. For example, five spatialmerge candidates may be selected from five merge candidates, and onetemporal merge candidate may be selected from two temporal mergecandidates.

Alternatively, the maximum number of merge candidates may be predefinedin the encoder and the decoder. For example, the maximum number of mergecandidates may be two, three, four, five, or six. Alternatively, themaximum number of merge candidates may be determined based on at leastone of whether merge with MVD (MMVD) is performed, whether combinedprediction is performed, or whether triangular partitioning isperformed.

If the number of merge candidates included in a merge candidate list issmaller than the maximum number of merge candidates, a merge candidateincluded in a second merge candidate list may be added to the mergecandidate list.

The second merge candidate list may include a merge candidate derivedbased on the motion information of a block encoded/decoded by interprediction before a current block. In an example, if motion compensationfor a block whose an encoding mode is inter prediction is performed, amerge candidate derived based on the motion information of the block maybe added to the second merge candidate list. If encoding/decoding of acurrent block is completed, the motion information of a current blockmay be added to the second merge candidate list for the inter predictionof the subsequent block.

The second merge candidate list may be initialized in a unit of a CTU, atile or a slice. The maximum number of merge candidates which may beincluded in the second merge candidate list may be predefined in anencoder and a decoder. Alternatively, information representing themaximum number of merge candidates which may be included in the secondmerge candidate list may be signaled through a bitstream.

The indexes of merge candidates included in the second merge candidatelist may be determined based on the order added to the second mergecandidate list. In an example, an index assigned to a N-th mergecandidate added to the second merge candidate list may have a valuesmaller than an index assigned to a N+1-th merge candidate added to thesecond merge candidate list. For example, an index of the N+1-th mergecandidate may be set to be a value increased by 1 to an index of theN-th merge candidate. Alternatively, an index of the N-th mergecandidate may be set to be an index of the N+1-th merge candidate andthe value of an index of the N-th merge candidate may be subtracted by1.

Alternatively, an index assigned to the N-th merge candidate added tothe second merge candidate list may have a value larger than an indexassigned to the N+1-th merge candidate added to the second mergecandidate list. For example, an index of the N-th merge candidate may beset to be an index of the N+1-th merge candidate and the value of anindex of the N-th merge candidate may be increased by 1.

Based on whether motion information of a block that motion compensationis performed is the same as motion information of a merge candidateincluded in the second merge candidate list, whether a merge candidatederived from the block is added to the second merge candidate list maybe determined. In an example, when a merge candidate with the samemotion information as the block is included in the second mergecandidate list, a merge candidate derived based on the motioninformation of the block may not be added to the second merge candidatelist. Alternatively, when a merge candidate with the same motioninformation as the block is included in the second merge candidate list,the merge candidate may be deleted from the second merge candidate listand a merge candidate derived based on the motion information of theblock may be added to the second merge candidate list.

When the number of merge candidates included in the second mergecandidate list is the same as the maximum number of merge candidates, amerge candidate with the lowest index or a merge candidate with thehighest index may be deleted from the second merge candidate list and amerge candidate derived based on the motion information of the block maybe added to the second merge candidate list. In other words, afterdeleting the oldest merge candidate among merge candidates included inthe second merge candidate list, a merge candidate derived based on themotion information of the block may be added to the second mergecandidate list.

When the number of merge candidates included in a merge candidate listdoes not reach the maximum number of merge candidates yet, a combinedmerge candidate obtained by combining two or more merge candidates or amerge candidate having a (0,0) motion vector (zero motion vector) may beincluded in the merge candidate list.

Alternatively, an average merge candidate striking an average of amotion vector of two or more merge candidates may be added to a mergecandidate list. An average merge candidate may be derived by striking anaverage of a motion vector of two or more merge candidates included in amerge candidate list. In an example, when a first merge candidate and asecond merge candidate are added to a merge candidate list, an averageof a motion vector of the first merge candidate and a motion vector ofthe second merge candidate may be calculated so as to obtain an averagemerge candidate. In detail, an L0 motion vector of an average mergecandidate may be derived by calculating an average of an L0 motionvector of the first merge candidate and an L0 motion vector of thesecond merge candidate, and an L1 motion vector of the average mergecandidate may be derived by calculating an average of an L1 motionvector of the first merge candidate and an L1 motion vector of thesecond merge candidate. When bi-directional prediction is applied to anyone of a first merge candidate and a second merge candidate, anduni-directional prediction is performed to the other one, a motionvector of the bi-directional merge candidate may be set as it is to anL0 motion vector or L1 motion vector of an average merge candidate. Inan example, when L0 directional and L1 directional predictions areperformed on a first merge candidate, but L0 directional prediction isperformed on a second merge candidate, an L0 motion vector of an averagemerge candidate may be derived by calculating an average of an L0 motionvector of the first merge candidate and an L0 motion vector of thesecond merge candidate. Meanwhile, an L1 motion vector of the averagemerge candidate may be derived as an L1 motion vector of the first mergecandidate.

When a reference picture of a first merge candidate differs with asecond merge candidate, a motion vector of the first merge candidate orsecond merge candidate may be scaled according to a distance (that is,POC difference) between reference pictures of respective mergecandidates and a current picture. For example, after scaling a motionvector of a second merge candidate, an average merge candidate may bederived by calculating an average of a motion vector of a first mergecandidate and the scaled motion vector of the second merge candidate.Herein, priorities may be set on the basis of a value of a referencepicture index of each merge candidate, a distance between a referencepicture of each merge candidate and a current block, or whether or notbi-directional prediction is applied, and scaling may be applied to amotion vector of a merge candidate having high (or low) priority.

A reference picture index of an average merge candidate may be set toindicate a reference picture at a specific position within a referencepicture list. In an example, a reference picture index of an averagemerge candidate may indicate the first or last reference picture withina reference picture list. Alternatively, a reference picture index of anaverage merge candidate may be set to be identical to a referencepicture index of a first merge candidate or second merge candidate. Inan example, when a reference picture index of a first merge candidate isidentical with a second merge candidate, a reference picture index of anaverage merge candidate may be set to be identical to a referencepicture index of the first merge candidate and the second mergecandidate. When a reference picture index of a first merge candidatediffers with a second merge candidate, priorities may be set on thebasis of a value of a reference picture index of each merge candidate, adistance between a reference picture of each merge candidate with thecurrent block, or whether or not bi-directional prediction is applied,and a reference picture index of a merge candidate with high (or low)priority may be set as a reference picture index of an average mergecandidate. In an example, when bi-directional prediction is applied to afirst merge candidate, and uni-directional prediction is applied to asecond merge candidate, a reference picture index of the first mergecandidate to which bi-directional prediction is applied may bedetermined as a reference picture index of an average merge candidate.

On the basis of priorities between combinations of merge candidates, thesequence of the combinations for generating an average merge candidatemay be determined. The priorities may be predefined in the encoder andthe decoder. Alternatively, the sequence of the combinations may bedetermined on the basis of whether bi-directional prediction of a mergecandidate is performed. For example, a combination of merge candidatesencoded using bi-directional prediction may be set to have a higherpriority that a combination of merge candidates encoded usinguni-directional prediction. Alternatively, the sequence of thecombinations may be determined on the basis of a reference picture of amerge candidate. For example, a combination of merge candidates havingthe same reference picture may have a higher priority than a combinationof merge candidates having different reference pictures.

A merge candidate may be included in a merge candidate list according topredefined priority. A merge candidate with high priority may beassigned with a small index value. In an example, a spatial mergecandidate may be added to a merge candidate list before than a temporalmerge candidate. In addition, spatial merge candidates may be added to amerge candidate list in an order of a spatial merge candidate of a leftneighboring block, a spatial merge candidate of a top neighboring block,a spatial merge candidate of a block adjacent to a top-right corner, aspatial merge candidate of a block adjacent to a bottom-left corner, anda spatial merge candidate of a block adjacent to a top-left corner.Alternatively, it may be set such that a spatial merge candidate derivedfrom a neighboring block adjacent to a top-left corner of a currentblock (B2 of FIG. 11) is added to a merge candidate list later than atemporal merge candidate.

In another example, priorities between merge candidates may bedetermined according to a size or shape of a current block. In anexample, when a current block has a rectangle shape where a width isgreater than a height, a spatial merge candidate of a left neighboringblock may be added to a merge candidate list before than a spatial mergecandidate of a top neighboring block. On the other hand, when a currentblock has a rectangle shape where a height is greater than a width, aspatial merge candidate of a top neighboring block may be added to amerge candidate list before than a spatial merge candidate of a leftneighboring block.

In another example, priorities between merge candidates may bedetermined according to motion information of respective mergecandidates. In an example, a merge candidate having bi-directionalmotion information may have priority higher than a merge candidatehaving uni-directional motion information. Accordingly, a mergecandidate having bi-directional motion information may be added to amerge candidate list before than a merge candidate havinguni-directional motion information.

In another example, a merge candidate list may be generated according topredefined priority, and then merge candidates may be rearranged.Rearranging may be performed on the basis of motion information of mergecandidates. In an example, rearranging may be performed on the basis ofwhether or not a merge candidate has bi-directional motion information,a size of a motion vector, precision of a motion vector, or a POCdifference between a current picture and a reference picture of a mergecandidate. In detail, a merge candidate having bi-directional motioninformation may be rearranged to have priority higher than a mergecandidate having uni-directional motion information. Alternatively, amerge candidate having a motion vector with a precision value of afractional-pel may be rearranged to have priority higher than a mergecandidate having a motion vector with a precision of an integer-pel.

When the merge candidate list is generated, at least one of mergecandidates included in the merge candidate list may be specified on thebasis of a merge candidate index S1040.

Motion information of the current block may be set to be identical tomotion information of the merge candidate specified by the mergecandidate index 51050. In an example, when a spatial merge candidate isselected by the merge candidate index, motion information of the currentblock may be set to be identical to motion information of the spatialneighboring block. Alternatively, when a temporal merge candidate isselected by the merge candidate index, motion information of the currentblock may be set to be identical to motion information of the temporalneighboring block.

FIG. 14 is a diagram showing a process of deriving motion information ofa current block when an AMVP mode is applied to the current block.

When an AMVP mode is applied to a current block, at least one of aninter-prediction direction of the current block, and a reference pictureindex may be decoded from a bitstream S1410. In other words, when anAMVP mode is applied, at least one of an inter-prediction direction ofthe current block, and a reference picture index may be determined onthe basis of information encoded through a bitstream.

A spatial motion vector candidate may be determined on the basis of amotion vector of a spatial neighboring block of the current block S1420.The spatial motion vector candidate may include at least one of a firstspatial motion vector candidate derived from a top neighboring block ofthe current block, and a second spatial motion vector candidate derivedfrom a left neighboring block of the current block. Herein, the topneighboring block may include at least one of blocks adjacent to a topand a top-right corner of the current block, and the left neighboringblock of the current block includes at least one of blocks adjacent to aleft and a left-bottom corner of the current block. The block adjacentto the left-top corner of the current block may be used as the topneighboring block or may be used as the left neighboring block.

Alternatively, a spatial motion vector candidate may be derived from aspatial non-neighboring block that is not adjacent to a current block.In an example, a spatial motion vector candidate of a current block maybe derived by using at least one of: a block positioned at the samevertical line with a block adjacent to a top, top-right corner, ortop-left corner of the current block; a block positioned at the samehorizontal line with a block adjacent to a left, bottom-left corner, ortop-left corner of the current block; and a block positioned at the samediagonal line with a block adjacent to a corner of the current block.When a spatial neighboring block is not available, a spatial motionvector candidate may be derived by using a spatial non-neighboringblock.

In another example, at least two spatial motion vector candidates may bederived by using a spatial neighboring block and spatial non-neighboringblocks. In an example, a first spatial motion vector candidate and asecond spatial motion vector candidate may be derived by usingneighboring blocks adjacent to a current block. Meanwhile, a thirdspatial motion vector candidate and/or a fourth spatial motion vectorcandidate may be derived on the basis of blocks that are not adjacent tothe current block but adjacent to the above neighboring blocks.

When the current block differs in a reference picture with the spatialneighboring block, a spatial motion vector may be obtained by performingscaling for a motion vector of the spatial neighboring block. A temporalmotion vector candidate may be determined on the basis of a motionvector of the temporal neighboring block of the current block S1430.When the current block differs in a reference picture with the temporalneighboring block, a temporal motion vector may be obtained byperforming scaling on a motion vector of the temporal neighboring block.Herein, when the number of spatial motion vector candidates is equal toor smaller than a predetermined number, a temporal motion vectorcandidate may be derived.

A motion vector candidate list including the spatial motion vectorcandidate and the temporal motion vector candidate may be generatedS1440.

When the motion vector candidate list is generated, at least one ofmotion vector candidates included in the motion vector candidate listmay be specified on the basis of information specifying at least one ofthe motion vector candidate list S1450.

The motion vector candidate specified by the information may be set as aprediction value of a motion vector of the current block, and the motionvector of the current block may be obtained by adding a residual valueof a motion vector to the prediction value of the motion vector S1460.Herein, the residual value of the motion vector may be parsed through abitstream.

When the motion information of the current block is obtained, motioncompensation for the current block may be performed on the basis of theobtained motion information S920. In detail, motion compensation for thecurrent block may be performed on the basis of an inter-predictiondirection, a reference picture index, and a motion vector of the currentblock. An inter prediction direction represents whether a L0-prediction,a L1-prediction or a bi-prediction is performed. When a current block isencoded by a bi-prediction, the prediction block of a current block maybe obtained based on the weighted sum operation or average operation ofa L0 reference block and a L1 reference block.

When a prediction sample is obtained by performing motion compensation,the current block may be reconstructed on the basis of the generatedprediction sample. In detail, a reconstructed sample may be obtained byadding a prediction sample of a current block and a residual sample.

As in the above-described example, on the basis of motion information ofthe block encode/decoded using inter prediction before the currentblock, a merge candidate of the current block may be derived. Forexample, on the basis of motion information of a neighboring block at apredefined position adjacent to the current block, a merge candidate ofthe current block may be derived. Examples of the neighboring block mayinclude at least one among a block adjacent to the left of the currentblock, a block adjacent to the top of the current block, a blockadjacent to the top left corner of the current block, a block adjacentto the top right corner of the current block, and a block adjacent tothe bottom left corner of the current block.

A merge candidate of the current block may be derived on the basis ofmotion information of a block other than the neighboring block. Forconvenience of description, a neighboring block at a predefined positionadjacent to the current block is referred to as a first merge candidateblock, and a block at a different position from the first mergecandidate block is referred to as a second merge candidate block.

The second merge candidate block may include at least one of a blockencoded/decoded using inter prediction before a current block, a blockadjacent to the first merge candidate block or a block positioned on thesame line as the first merge candidate block. FIG. 15 shows the secondmerge candidate block adjacent to the first merge candidate block andFIG. 16 shows the second merge candidate block positioned on the sameline as the first merge candidate block.

When the first merge candidate block is unavailable, a merge candidatederived on the basis of motion information of the second merge candidateblock is added to a merge candidate list. Alternatively, even though atleast one among a spatial merge candidate and a temporal merge candidateis added to a merge candidate list, when the number of merge candidatesincluded in the merge candidate list is smaller than the maximum numberof merge candidates, a merge candidate derived on the basis of motioninformation of the second merge candidate block is added to the mergecandidate list.

FIG. 15 is a diagram illustrating an example of deriving a mergecandidate from a second merge candidate block when a first mergecandidate block is unavailable.

When a first merge candidate block AN (herein, N ranges from 0 to 4) isunavailable, a merge candidate of the current block is derived on thebasis of motion information of a second merge candidate block BM(herein, M ranges from 0 to 6). That is, a merge candidate of thecurrent block may be derived by replacing the unavailable first mergecandidate block with the second merge candidate block.

Among the blocks adjacent to the first merge candidate block, the blockplaced in a predefined direction from the first merge candidate blockmay be set as a second merge candidate block. The predefined directionmay be a leftward direction, a rightward direction, an upward direction,a downward direction, or a diagonal direction. The predefine directionmay be set for each first merge candidate block. For example, apredefined direction of the first merge candidate block adjacent to theleft of the current block may be a leftward direction. A predefineddirection of the first merge candidate block adjacent to the top of thecurrent block may be an upward direction. A predefined direction of thefirst merge candidate block adjacent to the corner of the current blockmay include at least one of a leftward direction, an upward direction,or a diagonal direction.

For example, when A0 adjacent to the left of the current block isunavailable, a merge candidate of the current block is derived on thebasis of B0 adjacent to A1. When A1 adjacent to the top of the currentblock is unavailable, a merge candidate of the current block is derivedon the basis of B1 adjacent to A1. When A2 adjacent to the top rightcorner of the current block is unavailable, a merge candidate of thecurrent block is derived on the basis of B2 adjacent to A2. When A3adjacent to the bottom left corner of the current block is unavailable,a merge candidate of the current block is derived on the basis of B3adjacent to A3. When A4 adjacent to the top left corner of the currentblock is unavailable, a merge candidate of the current block is derivedon the basis of at least one among B4 to B6 adjacent to A4.

The example shown in FIG. 15 is only for describing an embodiment of thepresent invention, and does not limit the present invention. A positionof the second merge candidate block may be set different from the sampleshown in FIG. 15. For example, the second merge candidate block adjacentto the first merge candidate block adjacent to the left of the currentblock may be positioned in an upward direction or downward direction ofthe first merge candidate block. Alternatively, the second mergecandidate block adjacent to the first merge candidate block adjacent tothe top of the current block may be positioned in a leftward directionor rightward direction of the first merge candidate block.

FIG. 16 is a diagram showing an example of deriving a merge candidatefrom the second merge candidate block positioned on the same line as thefirst merge candidate block.

A block positioned on the same line as the first merge candidate blockmay include at least one of a block positioned on the same horizontalline as the first merge candidate block, a block positioned on the samevertical line as the first merge candidate block or a block positionedon the same diagonal line as the first merge candidate block. They-coordinate position of blocks positioned on the same horizontal lineare the same. The x-coordinate position of blocks positioned on the samevertical line are the same. A difference value between the x-coordinatepositions of blocks positioned on the same diagonal line is the same asa difference value between the y-coordinate positions.

It is assumed that the top-left sample of a current block is positionedat (0,0) and the width and height of a current block is W and H,respectively. In FIG. 18, it was shown that the position of the secondmerge candidate blocks (e.g., B4, C6) positioned on the same verticalline as the first merge candidate block is determined based on arightmost block at the top of a coding block (e.g., a block A1 includinga coordinate (W−1, −1)). In addition, in FIG. 18, it was shown that theposition of the second merge candidate blocks (e.g., B1, C1) positionedon the same horizontal line as the first merge candidate block isdetermined based on the lowest block at the left of a coding block(e.g., a block A0 including a coordinate (−1, H−1)).

In another example, the position of the second merge candidate blocksmay be determined based on the leftmost block at the top of a codingblock (e.g., a block including a coordinate (0, −1)) or a blockpositioned at the top center of a coding block (e.g., a block includinga coordinate (W/2, −1)). In addition, the position of the second mergecandidate blocks may be determined based on the topmost block at theleft of a coding block (e.g., a block including a coordinate (−1, 0)) ora block positioned at the left center of a coding block (e.g., a blockincluding a coordinate (−1, H/2)).

In another example, when there are a plurality of top neighboring blocksadjacent to the top of a current block, the second merge candidate blockmay be determined by using all or some of a plurality of top neighboringblocks. In an example, the second merge candidate block may bedetermined by using a block at a specific position (e.g., at least oneof a top neighboring block positioned at the leftmost side, a topneighboring block positioned at the rightmost side or a top neighboringblock positioned at the center) among a plurality of top neighboringblocks. The number of top neighboring blocks used to determine thesecond merge candidate block among a plurality of top neighboring blocksmay be 1, 2, 3 or more. In addition, when there are a plurality of leftneighboring blocks adjacent to the left of a current block, the secondmerge candidate block may be determined by using all or some of aplurality of left neighboring blocks. In an example, the second mergecandidate block may be determined by using a block at a specificposition (e.g., at least one of a left neighboring block positioned atthe bottommost side, a left neighboring block positioned at the topmostside or a left neighboring block positioned at the center) among aplurality of left neighboring blocks. The number of left neighboringblocks used to determine the second merge candidate block among aplurality of left neighboring blocks may be 1, 2, 3 or more.

According to the size and/or shape of a current block, the positionand/or number of top neighboring blocks and/or left neighboring blocksused to determine the second merge candidate block may be differentlydetermined. In an example, when the size of a current block is greaterthan a threshold value, the second merge candidate block may bedetermined based on a top center block and/or a left center block. Onthe other hand, when the size of a current block is smaller than athreshold value, the second merge candidate block may be determinedbased on a top rightmost block and/or a left bottommost block. Athreshold value may be an integer such as 8, 16, 32, 64 or 128.

The first merge candidate list and the second merge candidate list maybe constructed and motion compensation of the current block may beperformed based on at least one of the first merge candidate list or thesecond merge candidate list.

The first merge candidate list may include at least one of a spatialmerge candidate derived on the basis of motion information of aneighboring block at a predefined position adjacent to the currentblock, or a temporal merge candidate derived on the basis of motioninformation of a co-located block.

The second merge candidate list may include a merge candidate derived onthe basis of the motion information of the second merge candidate block.

As an embodiment of the present invention, the first merge candidatelist may be constructed including a merge candidate derived from thefirst merge candidate block, and the second merge candidate list may beconstructed including a merge candidate derived from the second mergecandidate block. In an example, in the example shown in FIG. 15, mergecandidates derived from blocks A0 to A4 may be added to the first mergecandidate list, and merge candidates derived from blocks B0 to B6 may beadded to the second merge candidate list. In an example, in the exampleshown in FIG. 16, merge candidates derived from blocks A0 to A4 may beadded to the first merge candidate list and merge candidates derivedfrom blocks B0 to B5, C0 to C7 may be added to the second mergecandidate list.

Alternatively, the second merge candidate list may include a mergecandidate derived on the basis of motion information of a block that isencoded/decoded using inter prediction before the current block. Forexample, when motion compensation for a block of which an encoding modeis inter prediction is performed, a merge candidate derived on the basisof motion information of the block is added to the second mergecandidate list. When encoding/decoding of the current block iscompleted, motion information of the current block is added to thesecond merge candidate list for inter prediction of the subsequentblock.

Indexes of the merge candidates included in the second merge candidatelist may be determined on the basis of the order in which the mergecandidates are added to the second merge candidate list. For example, anindex allocated to the N-th merge candidate added to the second mergecandidate list may have a lower value than an index allocated to theN+1-th merge candidate added to the second merge candidate list. Forexample, an index of the N+1-th merge candidate may be set to have ahigher value by one than an index of the N-th merge candidate.Alternatively, an index of the N-th merge candidate may be set to anindex of the N+1-th merge candidate, and a value of the index of theN-th merge candidate subtract is decreased by one.

Alternatively, an index allocated to the N-th merge candidate added tothe second merge candidate list may have a higher value than an indexallocated to the N+1-th merge candidate added to the second mergecandidate list. For example, an index of the N-th merge candidate may beset to an index of the N+1-th merge candidate, and a value of the indexof the N-th merge candidate subtract is increased by one.

On the basis of whether motion information of a block subjected tomotion compensation is the same as motion information of the mergecandidate included in the second merge candidate list, it may bedetermined whether to add a merge candidate derived from the block tothe second merge candidate list. For example, when the merge candidatehaving the same motion information as the block is included in thesecond merge candidate list, a merge candidate derived on the basis ofthe motion information of the block is not added to the second mergecandidate list. Alternatively, when the merge candidate having the samemotion information as the block is included in the second mergecandidate list, the merge candidate is deleted from the second mergecandidate list and a merge candidate derived on the basis of the motioninformation of the block is added to the second merge candidate list.

When the number of merge candidates included in the second mergecandidate list is the same as the maximum number of merge candidates,the merge candidate having the lowest index or the merge candidatehaving the highest index is detected from the second merge candidatelist and a merge candidate derived on the basis of the motioninformation of the block is added to the second merge candidate list.That is, after deleting the oldest merge candidate among the mergecandidates included in the second merge candidate list, a mergecandidate derived on the basis of the motion information of the blockmay be added to the second merge candidate list.

The second merge candidate list may be initialized in a unit of a CTU, atile or a slice. In other words, a block included in a CTU, a tile or aslice different from a current block may be set to be unavailable as thesecond merge candidate block. The maximum number of merge candidateswhich may be included in the second merge candidate list may bepredefined in an encoder and a decoder. Alternatively, informationrepresenting the maximum number of merge candidates which may beincluded in the second merge candidate list may be signaled through abitstream.

Either the first merge candidate list or the second merge candidate listmay be selected and inter prediction of the current block may beperformed using the selected merge candidate list. Specifically, on thebasis of index information, any one of the merge candidates included inthe merge candidate list may be selected and motion information of thecurrent block may be acquired from the merge candidate.

Information specifying either the first merge candidate list or thesecond merge candidate list may be signaled through a bitstream. Thedecoder may select either the first merge candidate list or the secondmerge candidate list on the basis of the information.

Alternatively, among the first merge candidate list and the second mergecandidate list, the merge candidate list including a larger number ofavailable merge candidates may be selected.

Alternatively, either the first merge candidate list or the second mergecandidate list may be selected on the basis of at least one among thesize, the shape, and the partition depth of the current block.

Alternatively, a merge candidate list configured by adding (orappending) the other to any of the first merge candidate list and thesecond merge candidate list.

For example, inter prediction may be performed on the basis of a mergecandidate list including at least one merge candidate included in thefirst merge candidate list, and at least one merge candidate included inthe second merge candidate list.

For example, a merge candidate included in the second merge candidatelist may be added to the first merge candidate list. Alternatively, amerge candidate included in the first merge candidate list may be addedto the second merge candidate.

When the number of merge candidates included in the first mergecandidate list is smaller than the maximum number, or when the firstmerge candidate block is unavailable, a merge candidate included in thesecond merge candidate list is added to the first merge candidate list.

Alternatively, when the first merge candidate block is unavailable, themerge candidate derived from a block adjacent to the first mergecandidate block among the merge candidates included in the second mergecandidate list is added to the first merge candidate list. Referring toFIG. 15, when A0 is unavailable, a merge candidate derived on the basisof motion information of B0 among the merge candidates included in thesecond merge candidate list is added to the first merge candidate list.When A1 is unavailable, a merge candidate derived on the basis of motioninformation of B1 among the merge candidates included in the secondmerge candidate list is added to the first merge candidate list. When A2is unavailable, a merge candidate derived on the basis of motioninformation of B2 among the merge candidates included in the secondmerge candidate list is added to the first merge candidate list. When A3is unavailable, a merge candidate derived on the basis of motioninformation of B3 among the merge candidates included in the secondmerge candidate list is added to the first merge candidate list. When A4is unavailable, a merge candidate derived on the basis of motioninformation of B4, B5, or B6 among the merge candidates included in thesecond merge candidate list is added to the first merge candidate list.

Alternatively, a merge candidate to be added to the first mergecandidate list may be determined according to the priorities of themerge candidates included in the second merge candidate list. Thepriorities may be determined based on an index value assigned to eachmerge candidate. For example, when the number of merge candidatesincluded in the first merge candidate list is smaller than the maximumnumber, or when the first merge candidate block is unavailable, themerge candidate having the smallest index value or the merge candidatehaving the largest index value among the merge candidates included inthe second merge candidate list is added to the first merge candidatelist.

When a merge candidate having the same motion information as a mergecandidate with the highest priority among merge candidates included inthe second merge candidate list is included in the first merge candidatelist, the merge candidate with the highest priority may not be added tothe first merge candidate list. In addition, whether a merge candidatewith a next priority (e.g., a merge candidate to which an index valuelarger than an index value assigned to a merge candidate with thehighest priority by 1 is assigned or a merge candidate to which an indexvalue smaller than an index value assigned to a merge candidate with thehighest priority by 1 is assigned) may be added to the first mergecandidate list may be determined.

Alternatively, a merge candidate list including a merge candidatederived on the basis of motion information of the first merge candidateblock, and a merge candidate derived on the basis of motion informationof the second merge candidate block may be generated. The mergecandidate list may be a combination of the first merge candidate listand the second merge candidate list.

For example, according to a predetermined order of searching, a mergecandidate list may be generated by searching for the first mergecandidate block and the second merge candidate block.

FIGS. 17 to 20 are diagrams illustrating the order of searching formerge candidate blocks.

FIGS. 17 to 20 shows the order of searching for merge candidates asfollows.

A0→A1→A2→A3→A4→B0→B1→B2→B3→B4→(B5)→(B6)

Only when a block B4 is unavailable or when the number of mergecandidates included in the merge candidate list is equal to or smallerthan a preset number, searching for blocks B5 and B6 takes place.

The different order of searching from the examples shown in FIGS. 17 to20 may be set.

A combined merge candidate list including at least one merge candidateincluded in the first merge candidate list, and at least one mergecandidate included in the second merge candidate list may be generated.For example, the combined merge candidate list may include N of mergecandidates included in the first merge candidate list, and M of mergecandidates included in the second merge candidate list. The letters Nand M may denote the same number or different numbers. Alternatively, atleast one among N and M may be determined on the basis of at least oneamong the number of merge candidates included in the first mergecandidate list and the number of merge candidates included in the secondmerge candidate list. Alternatively, information for determining atleast one among N and M may be signaled through a bitstream. Any oneamong N and M may be derived by subtracting the other from the maximumnumber of merge candidates in the combined merge candidate list.

Merge candidates to be added to the combined merge candidate list may bedetermined according to a predefined priority. The predefined prioritymay be determined on the basis of indexes allocated to the mergecandidates.

Alternatively, a merge candidate to be added to the combined mergecandidate list may be determined on the basis of association betweenmerge candidates. For example, when A0 included in the first mergecandidate list is added to the combined merge candidate list, a mergecandidate (for example, B0) at a position adjacent to A0 is not added toa combined merge list.

When the number of the merge candidates included in the first mergecandidate list is smaller than N, more than M merge candidates among themerge candidates included in the second merge candidate list are addedto the combined merge candidate list. For example, when N is four and Mis two, four of the merge candidates included in the first mergecandidate list are added to the combined merge candidate list, and twoof the merge candidates included in the second merge candidate list areadded to the combined merge candidate list. When the number of the mergecandidates included in the first merge candidate list is smaller thanfour, two or more merge candidates among the merge candidates includedin the second merge candidate list are added to the combined mergecandidate list. When the number of the merge candidates included in thesecond merge candidate list is smaller than two, four or more of themerge candidates included in the first merge candidate list are added tothe combined merge candidate list.

That is, the value of N or M may be adjusted according to the number ofmerge candidates included in each merge candidate list. By adjusting thevalue of N or M, the total number of merge candidates included in thecombined merge candidate list may be fixed. When the total number ofmerge candidates included in the combined merge candidate list issmaller than the maximum number of merge candidates, a combined mergecandidate, an average merge candidate, or a zero motion vector candidateis added.

A motion compensation of a current block may be performed by using atleast one of merge candidates included in a first merge candidate listand a second merge candidate list. An encoder may encode indexinformation for specifying any one of a plurality of merge candidates.In an example, ‘merge_idx’ may specify any one of a plurality of mergecandidates. In an example, Table 1 represents a merge index of eachmerge candidate derived from the first merge candidate blocks and thesecond merge candidate blocks shown in FIG. 16.

TABLE 1 Merge Candidate Merge Index (merge_idx) A1 0 A2 1 A3 2 A4 3 B1 4B2 5 B3 6 B4 7 B5 8 C1 9 C2 10 C3 11 C4 12 C5 13 C6 14 C7 15

However, as the number of merge candidates included in a merge candidatelist increases, a codeword for encoding a merge index gets longer.Accordingly, a problem occurs that encoding/decoding efficiency isreduced. To reduce a length of a codeword, a merge index may bedetermined by using a prefix and a suffix. In an example, a merge indexmay be determined by using merge_idx_prefix representing the prefix ofthe merge index and merge_idx_suffix representing the suffix of themerge index.

Table 2 represents a merge index prefix value and a merge index suffixvalue for each merge index and Table 3 represents a process ofdetermining a merge index based on a merge index prefix value and amerge index suffix value.

TABLE 2 Merge Merge Index Merge Index Merge Index Candidate (merge_idx)Prefix Suffix A1 0 0 — A2 1 1 — A3 2 2 — A4 3 3 — B1 4 4 0 B2 5 4 1 B3 64 2 B4 7 4 3 B5 8 4 4 C1 9 5 0 C2 10 5 1 C3 11 5 2 C4 12 5 3 C5 13 5 4

TABLE 3 if merge_idx_prefix < 4  merge_idx = merge_idx_prefix else merge_idx = (merge_idx_prefix−3) << 2 + merge_idx_suffix

As shown in Tables 2 and 3, when a merge index prefix value is smallerthan a threshold value, a merge index may be set to be the same as themerge index prefix value. On the other hand, when a merge index prefixvalue is greater than a threshold value, a merge index may be determinedby subtracting a base value from the merge index prefix and adding amerge index suffix to a value shifting that result. The base value maybe the threshold value or a value obtained by subtracting 1 from thethreshold value.

Tables 2 and 3 shows that a threshold value is 4. The threshold valuemay be determined based on at least one of the number of mergecandidates included in a merge candidate list, the number of the secondmerge candidate blocks or the number of lines in which the second mergecandidate blocks are included. Alternatively, the threshold value may bepredefined in an encoder and a decoder.

Whether a prefix and a suffix are used to determine a merge index may bedetermined according to the number of merge candidates included in amerge candidate list or the maximum number of merge candidates which maybe included in the merge candidate list. In an example, when the maximumnumber of merge candidates which may be included in a merge candidatelist is greater than a threshold value, a merge index prefix and a mergeindex suffix used to determine a merge index may be signaled. On theother hand, when the maximum number of merge candidates is smaller thanthe threshold value, a merge index may be signaled.

A rectangular block may be partitioned into a plurality of triangularblocks. Merge candidates of the triangular blocks may be derived basedon a rectangular block including the triangular blocks. The triangularblocks may share the same merge candidate.

A merge index may be signaled for each triangular block. In this case,triangular blocks may be set not to use the same merge candidate. In anexample, a merge candidate used for a first triangular block may not beused as a merge candidate of a second triangular block. Accordingly, themerge index of the second triangular block may specify any one ofremaining merge candidates excluding a merge candidate selected for thefirst triangular block.

A merge candidate may be derived on the basis of a block having apredetermined shape or a predetermined size or larger. When the currentblock is not in a predetermined shape, or when the size of the currentblock is smaller than a predetermined size, a merge candidate of thecurrent block is derived on the basis of a block including the currentblock and being in the a predetermined shape or in the predeterminedsize or larger. The predetermined shape may be a square shape or anon-square shape.

When the predetermined shape is a square shape, a merge candidate for acoding unit in a non-square shape is derived on the basis of a codingunit in a square shape including the coding unit in the non-squareshape.

FIG. 21 is a diagram illustrating an example in which a merge candidateof a non-square block is derived on the basis of a square block.

A merge candidate of a non-square block may be derived on the basis of asquare block including the non-square block. For example, a mergecandidate of a coding block 0 in a non-square shape and a coding block 1in a non-square shape may be derived on the basis of a block in a squareshape including the coding block 0 and the coding block 1. That is, aposition of a spatial neighboring block may be determined on the basisof a position, a width/height, or a size of a block in a square shape. Amerge candidate of a coding block 0 and a coding block 1 may be derivedon the basis of at least one among spatial neighboring blocks A0, A1,A2, A3, and A4 adjacent to a block in a square shape.

A temporal merge candidate may be determined on the basis of a block ina square shape. That is, a temporal neighboring block may be determinedon the basis of a position, a width/height, or a size of a block in asquare shape. For example, a merge candidate of a coding block 0 and acoding block 1 may be derived on the basis of the temporal neighboringblock determined on the basis of the block in the square shape.

Alternatively, any one among a spatial merge candidate and a temporalmerge candidate may be derived on the basis of a square block, and theother merge candidate may be derived on the basis of a non-square block.For example, a spatial merge candidate of a coding block 0 may bederived on the basis of a square block, while a temporal merge candidateof the coding block 0 may be derived on the basis of the coding block 0.

Multiple blocks included in a block in a predetermined shape or apredetermined size or larger may share a merge candidate. For example,in the example shown in FIG. 21, at least one among a spatial mergecandidate and a temporal merge candidate of a coding block 0 and acoding block 1 may be the same.

The predetermined shape may be a non-square shape, such as 2N×N, N×2N,or the like. When the predetermined shape is a non-square shape, a mergecandidate of the current block may be derived on the basis of anon-square block including the current block. For example, when thecurrent block is in a 2N×n shape (herein, n is ½N), a merge candidate ofthe current block is derived on the basis of a non-square block in a2N×N shape. Alternatively, when the current block is in a n×2N shape, amerge candidate of the current block is derived on the basis of anon-square block in an N×2N shape.

Information indicating a predetermined shape or a predetermined size maybe signaled through a bitstream. For example, information indicating anyone among a non-square shape or a square shape may be signaled through abitstream.

Alternatively, a predetermined shape or a predetermined size may bedetermined according to a rule predefined in the encoder and thedecoder.

When a child node does not satisfy a predetermined condition, a mergecandidate of the child node is derived on the basis of a parent nodesatisfying the predetermined condition. Herein, the predeterminedcondition may include at least one among whether the block is a blockgenerated as a result of quad tree partitioning, whether exceeding thesize of the block, the shape of the block, and the picture boundarytakes place, and whether the difference in depth between the child nodeand the parent node is equal to or greater than a predetermined value.

For example, predetermined conditions may include whether the block is ablock generated as a result of quad tree partitioning, and whether theblock is a square shape coding block in a predetermined size or larger.When the current block is generated by binary tree partitioning ortriple tree partitioning, a merge candidate of the current block isderived on the basis of a high-level node block that includes thecurrent block and satisfies the predetermined conditions. When there isno high-level node block satisfying the predetermined conditions, amerge candidate of a current block is derived on the basis of thecurrent block, a block that includes the current block and is in apredetermined size or larger, or a high-level node block that includesthe current block and has the depth difference of one with the currentblock.

FIG. 22 is a diagram illustrating an example of deriving a mergecandidate on the basis of a high-level node block.

A block 0 and a block 1 are generated by partitioning a square block onthe basis of a binary tree. A merge candidate of the block 0 and theblock 1 may be derived on the basis of a neighboring block (that is, atleast one among A0, A1, A2, A3, and A4) that is determined on the basisof a high-level node block including the block 0 and the block 1. As aresult of this, the block 0 and the block 1 may use the same spatialmerge candidate.

A high-level node block including a block 2 and a block 3, and a block 4may be generated by partitioning a square block on the basis of a binarytree. In addition, the block 2 and the block 3 may be generated bypartitioning a block in a non-square shape on the basis of a binarytree. A merge candidate of the block 2, the block 3, and the block 4 innon-square shapes may be derived on the basis of a high-level node blockincluding the same. That is, a merge candidate may be derived on thebasis of a neighboring block (for example, at least one among B0, B1,B2, B3, and B4) that is determined on the basis of a position, awidth/height, or a size of a square block including the block 2, theblock 3, and the block 4. As a result of this, the block 2, the block 3,and the block 4 may use the same spatial merge candidate.

A temporal merge candidate for a block in a non-square shape may bederived on the basis of a high-level node block. For example, a temporalmerge candidate for the block 0 and the block 1 may be derived on thebasis of a square block including the block 0 and the block 1. Atemporal merge candidate for the block 2, the block 3, and the block 4may be derived on the basis of a square block including the block 2, theblock 3, and the block 4. In addition, the same temporal merge candidatederived from a temporal neighboring block determined on a per-quad treeblock basis may be used.

Low-level node blocks included in a high-level node block may share atleast one among a spatial merge candidate and a temporal mergecandidate. For example, the low-level node blocks included in thehigh-level node block may use the same merge candidate list.

Alternatively, at least one among a spatial merge candidate and atemporal merge candidate may be derived on the basis of a low-level nodeblock, and the other may be derived on the basis of a high-level nodeblock. For example, a spatial merge candidate for the block 0 and theblock 1 may be derived on the basis of the high-level node block.However, a temporal merge candidate for the block 0 may be derived onthe basis of the block 0, and a temporal merge candidate for the block 1may be derived on the basis of the block 1.

Alternatively, when the number of samples that a low-level node blockincludes is smaller than a predefined number, a merge candidate isderived on the basis of a high-level node block including the predefinednumber or more of samples. For example, when at least one of thefollowing conditions is satisfied: a case where at least one oflow-level node blocks generated on the basis of at least one among quadtree partitioning, binary tree partitioning, and triple treepartitioning is smaller than a preset size; a case where at least one ofthe low-level node blocks is a non-square block; a case where ahigh-level node block does not exceed a picture boundary; and a casewhere a width or height of a high-level node block is equal to orgreater than a predefined value, a merge candidate is derived on thebasis of a high-level node block in a square or non-square shapeincluding a predefined number of more of samples (for example, 64, 128,or 256 samples). The low-level node blocks included in the high-levelnode block may share merge candidates derived on the basis of thehigh-level node block.

A merge candidate may be derived on the basis of any one of low-levelnode block, and the other low-level node blocks may be set to use themerge candidate. The low-level node blocks may be included in a block ina predetermined shape or a predetermined size or larger. For example,low-level node blocks may share a merge candidate list derived on thebasis of any one of the low-level node blocks. Information for alow-level node block that is the basis of derivation of the mergecandidate may be signaled through a bitstream. The information may beindex information indicating any one of low-level node blocks.Alternatively, the low-level node block that is the basis of derivationof the merge candidate may be determined on the basis of at least oneamong positions, sizes, shapes, and the scanning order of the low-levelnode blocks.

Information indicating whether low-level node blocks share a mergecandidate list derived on the basis of a high-level node block may besignaled through a bitstream. On the basis of the information, it may bedetermined whether a merge candidate of a block not in a predeterminedshape or a block in a size smaller than a predetermined size is derivedon the basis of a high-level node block including the block.Alternatively, according to a rule predefined in the encoder and thedecoder, it may be determined whether a merge candidate is derived onthe basis of a high-level node block.

When a neighboring block adjacent to the current block is present withina predefined region, it is determined that the neighboring block isunavailable as a spatial merge candidate. The predefined region may be aparallel-processing region defined for parallel processing betweenblocks. The parallel-processing region may be referred to as a mergeestimation region (MER). For example, when a neighboring block adjacentto the current block is included in the same merge estimation region asthe current block, it is determined that the neighboring block isunavailable. A shift operation may be performed so as to determinewhether the current block and the neighboring block are included in thesame merge estimation region. Specifically, on the basis of whether avalue obtained by shifting the position of the top left reference sampleof the current block is the same as a value obtained by shifting theposition of the top left reference sample of the neighboring block, itmay be determined whether the current block and the neighboring blockare included in the same merge estimation region.

FIG. 23 is a diagram illustrating an example of determining availabilityof a spatial neighboring block on the basis of a merge estimationregion.

In FIG. 23, it is shown that a merge estimation region is in a N×2Nshape.

A merge candidate of a block 1 may be derived on the basis of a spatialneighboring block adjacent to the block 1. The spatial neighboringblocks may include B0, B1, B2, B3, and B4. Herein, it may be determinedthat the spatial neighboring blocks B0 and B3 included in the same mergeestimation region as the block 1 is unavailable as merge candidates.Accordingly, a merge candidate of the block 1 may be derived from atleast one of the spatial neighboring blocks B1, B2, and B4 excluding thespatial neighboring blocks B0 and B3.

A merge candidate of a block 3 may be derived on the basis of a spatialneighboring block adjacent to the block 3. The spatial neighboringblocks may include C0, C1, C2, C3, and C4. Herein, it may be determinedthat the spatial neighboring block C0 included in the same mergeestimation region as the block 3 is unavailable as a merge candidate.Accordingly, a merge candidate of the block 3 may be derived from atleast one of the spatial neighboring blocks C1, C2, C3, and C4 excludingthe spatial neighboring block C0.

On the basis of at least one among a position, a size, a width, and aheight of a merge estimation region, a merge candidate of a blockincluded in the merge estimation region may be derived. For example, amerge candidate of multiple blocks included in a merge estimation regionmay be derived from at least one among a spatial neighboring block and atemporal neighboring block that are determined on the basis of at leastone among a position, a size, a width, and a height of the mergeestimation region. The blocks included in the merge estimation regionmay share the same merge candidate.

FIG. 24 is a diagram illustrating an example in which a merge candidateis derived on the basis of a merge estimation region.

When multiple coding units are included in a merge estimation region, amerge candidate of the multiple coding units may be derived on the basisof the merge estimation region. That is, by using the merge estimationregion as a coding unit, a merge candidate may be derived on the basisof the position, the size, or the width/height of the merge estimationregion.

For example, a merge candidate of a coding unit 0 (CU0) and a codingunit 1 (CU1) both in a (n/2)×N (herein, n is N/2) size and included in amerge estimation region in a (N/2)×N size may be derived on the basis ofthe merge estimation region. That is, a merge candidate of the codingunit 0 and the coding unit 1 may be derived from at least one ofneighboring blocks C0, C1, C2, C3, and C4 adjacent to the mergeestimation region.

For example, a merge candidate of a coding unit 2 (CU2), a coding unit 3(CU3), a coding unit 4 (CU4), and a coding unit 5 (CU5) in an n×n sizeincluded in a merge estimation region in an N×N size may be derived onthe basis of the merge estimation region. That is, a merge candidate ofthe coding unit 2, the coding unit 3, the coding unit 4, and the codingunit 5 may be derived from at least one of the neighboring blocks C0,C1, C2, C3, and C4 adjacent to the merge estimation region.

The shape of the merge estimation region may be a square shape or anon-square shape. For example, it may be determined that a coding unit(or prediction unit) in a square shape or a coding unit (or predictionunit) in a non-square shape is a merge estimation region. The ratiobetween the width and the height of the merge estimation region may belimited not to exceed a predetermined range. For example, the mergeestimation region is unable to have a non-square shape of which theratio between the width and the height exceeds two, or a non-squareshape of which the ratio between the width and the height is less than½. That is, the non-square merge estimation region may be in a 2N×N orN×2N shape. Information on a limit on the ratio between the width andthe height may be signaled through a bitstream. Alternatively, a limiton the ratio between the width and the height may be predefined in theencoder and the decoder.

At least one among information indicating the shape of the mergeestimation region, and information indicating the size of the mergeestimation region may be signaled through a bitstream. For example, atleast one among the information indicating the shape of the mergeestimation region, and the information indicating the size of the mergeestimation region may be signaled through a slice header, a tile groupheader, a picture parameter, or a sequence parameter.

The shape of the merge estimation region or the size of the mergeestimation region may be updated on a per-sequence basis, a per-picturebasis, a per-slice basis, a per-tile group basis, a per-tile basis, or aper-block (CTU) basis. When the shape of the merge estimation region orthe size of the merge estimation region is different from that of theprevious unit, information indicating a new shape of the mergeestimation region or a new size of the merge estimation region issignaled through a bitstream.

At least one block may be included in the merge estimation region. Theblock included in the merge estimation region may be in a square shapeor a non-square shape. The maximum number or the minimum number ofblocks that the merge estimation region is able to include may bedetermined. For example, three, four, or more CUs may be included in themerge estimation region. The determination may be based on informationsignaled through a bitstream. Alternatively, the maximum number or theminimum number of blocks that the merge estimation region is able toinclude may be predefined in the encoder and the decoder.

In at least one among a case where the number of blocks included in themerge estimation region is smaller than the maximum number, and a casewhere the number is larger than the minimum number, parallel processingof the blocks may be allowed. For example, when the number of blocksincluded in the merge estimation region is equal to or smaller than themaximum number, or when the number of blocks included in the mergeestimation region is equal to or larger than the minimum number, a mergecandidate of the blocks is derived on the basis of the merge estimationregion. When the number of blocks included in the merge estimationregion is larger than the maximum number, or when the number of blocksincluded in the merge estimation region is smaller than the minimumvalue, a merge candidate of each of the blocks is derived on the basisof the size, the position, the width, or the height of each of theblocks.

Information indicating the shape of the merge estimation region mayinclude a one-bit flag. For example, the syntax “isrectagular_mer_flag”may indicate that the merge candidate region in a square shape or anon-square shape. The isrectagular_mer_flag value of one may indicatethat the merge estimation region in a non-square shape, and theisrectagular_mer_flag value of zero may indicate that the mergeestimation region in a square shape.

When the information indicates that the merge estimation region in anon-square shape, information indicating at least one among the width,the height, and the ratio between the width and the height of the mergeestimation region is signaled through a bitstream. On the basis of this,the size and/or the shape of the merge estimation region may bedetermined.

Next, according to an embodiment of the present disclosure, an interprediction method based on an affine motion model will be described indetail.

A motion of an object may be classified into a translation motion, arotation motion and an affine motion. The translation motion representsa linear movement of the object. The rotation motion represents arotational movement of the object. The affine motion represents that amovement, a rotation or a displacement is different for each part of theobject.

FIG. 25 is a diagram illustrating a motion model.

FIG. 25 (a) shows a translation motion model. FIG. 25 (b) shows arotation motion model. FIG. 25 (c) shows an affine motion model.

In a translation motion model, a linear motion of an object may berepresented with a 2D coordinate and/or a motion vector (MVx, MVy).

In a rotation motion model, a motion of an object may be representedwith a predetermined rotation angle.

On the other hand, in an affine motion model, since a motion of anobject is nonlinear, so at least one of a movement, a rotation or adisplacement may be different for each part of an object. In the affinemotion model, the motion of the object may be represented based on amovement, a rotation or a adjustment of a displacement for each part ofthe object. For example, the affine motion model may be represented byusing a predetermined parameter (a-f) as in the following Equation 1.

x′=ax+by+e

y′=cx+dy+f  [Equation 1]

In Equation 1, x and y represent a position of a sample included in acurrent block and x′ and y′ represent a position of a sample included ina reference block. An affine motion vector (V_(x), V_(y)) which is aposition difference between the sample in the current block and thesample of the reference block may be represented as in Equation 2.

Vx=x−x′

Vy=y−y′  [Equation 2]

An affine motion vector (V_(x), V_(y)) may be derived as in thefollowing Equation 3 by using Equation 1 and Equation 2.

Vx=(1−a)z−by−e

Vy=(1−d)y−ex−f  [Equation 3]

A complex motion of an object can be expressed through an affine motionmodel. But, in the affine motion model, encoding/decoding efficiency maybe reduced as more parameters should be encoded to express a motion ofan object compared with a translation motion model or a rotation motionmodel. To solve the problem, when perform motion prediction and/ormotion compensation based on the affine motion model, a motion vector ofa current block corner may be used. Concretely, after partitioning acurrent block into a plurality of sub-blocks, a motion vector of eachpartitioned sub-block may be determined by using a plurality of cornermotion vectors. Hereinafter, an affine inter prediction method usingcorner motion vectors will be described in detail.

FIG. 26 is a diagram illustrating an affine motion model using cornermotion vectors.

In an affine motion model, a motion vector of at least one of a left-topcorner, a right-top corner, a left-bottom corner or a right-bottomcorner of a current block may be used. In an example, motion predictionand/or motion compensation for the current block may be performed byusing a motion vector v0 at the left-top corner of the current block anda motion vector v1 at the right-top corner of the current block.Alternatively, motion prediction and/or motion compensation for thecurrent block may be performed by using a motion vector v0 at theleft-top corner of the current block, a motion vector v1 at theright-top corner of the current block and a motion vector v2 at theleft-bottom corner of the current block.

When three corner motion vectors are used, it may be expressed that acurrent block is changed into a rectangle which has two sides formed byconnecting pixels in the reference picture indicated by the three cornermotion vectors of the current block. In other words, it may be estimatedthat the rectangle, which has two sides formed by connecting pixels inthe reference picture indicated by the three corner motion vectors, as areference block of the current block.

When three corner motion vectors are used, a motion vector of eachsub-block in a current block may be determined based on the followingEquation 4.

$\begin{matrix}{{v_{x} = {{\frac{\left( {v_{1x} - v_{0x}} \right)}{w}x} - {\frac{\left( {v_{2x} - v_{0x}} \right)}{w}y} + v_{0x}}}{v_{y} = {{\frac{\left( {v_{1y} - v_{0y}} \right)}{w}x} - {\frac{\left( {v_{2y} - v_{0y}} \right)}{w}y} + v_{0y}}}} & \left\lbrack {{Equation}\mspace{14mu} 4} \right\rbrack\end{matrix}$

In Equation 4, w and h represent a width and height of a current block,respectively. x and y represent a position of a sub-block. The positionof the sub-block may be a position of a predefined sample in thesub-block. The predefined sample may be a left-top sample, a left-bottomsample, a right-top sample or a center sample.

When three corner motion vectors are used, a motion vector of asub-block is derived based on a total of 6 motion vector parameters.Accordingly, affine inter prediction using three corner motion vectorsmay be referred to as a 6-parameter mode.

When two corner motion vectors are used, it may be expressed that acurrent block is changed into a rectangle which has a side formed byconnecting pixels in the reference picture indicated by the two cornermotion vectors of the current block. In other words, it may be estimatedthat the rectangle, which has a side formed by connecting pixels in thereference picture indicated by two corner motion vectors, as a referenceblock of the current block.

As in an example shown in FIG. 26, a relation of the following Equation5 is established between a motion vector of a left-bottom corner, amotion vector of t a left-top corner and a motion vector of a right-topcorner.

v _(2x) =v _(0x) +v _(0y) −v _(1y)

v _(2y) =−v _(0x) +v _(0y) +v _(1x)[Equation 5]

Based on the Equation 5, Equation 4 may be changed like the followingEquation 6.

$\begin{matrix}{{v_{x} = {{\frac{\left( {v_{1x} - v_{0x}} \right)}{w}x} - {\frac{\left( {v_{1y} - v_{0y}} \right)}{w}y} + v_{0x}}}{v_{y} = {{\frac{\left( {v_{1y} - v_{0y}} \right)}{w}x} - {\frac{\left( {v_{1x} - v_{0x}} \right)}{w}y} + v_{0y}}}} & \left\lbrack {{Equation}\mspace{14mu} 6} \right\rbrack\end{matrix}$

When two corner motion vectors are used, a motion vector of eachsub-block in a current block may be determined based on the Equation 6.

When two corner motion vectors are used, a motion vector of a sub-blockis derived based on a total of 4 motion vector parameters. Accordingly,affine inter prediction using two corner motion vectors may be referredto as a 4-parameter mode.

FIG. 27 is a diagram illustrating an example in which a motion vector isdetermined in a unit of a sub-block.

A current block may be partitioned into a plurality of sub-blocks and amotion vector of each sub-block may be obtained based on corner motionvectors. Based on the motion vector of the sub-block, motioncompensation for the sub-block may be performed.

The number of sub-blocks in a current block may be determined based on asize and/or shape of the current block. In this case, the size of thecurrent block may be represented with width, height, the sum of widthand height or the product of width and height. The shape of the currentblock may represent at least one of whether the current block is square,whether the current block is non-square, whether width is greater thanheight or a width and height ratio.

In an example, when a size of a current block is smaller than athreshold value, the current block may be partitioned into sub-blocks ofa first size. On the other hand, when a size of a current block isgreater than the threshold value, the current block may be partitionedinto sub-blocks of a second size. In this case, the threshold value maybe an integer such as 8, 16, 32 or 64. The first size may be 4×4, 8×8 or16×16 and the second size may be a value larger than the first size. Fora detailed example, when the size of the current block is 16×16, thecurrent block may be partitioned into four 8×8-sized sub-blocks. On theother hand, when the size of the current block is greater than 16×16(e.g., 32×32 or 64×64), the current block may be partitioned into16×16-sized sub-blocks.

Alternatively, sub-blocks in a fixed size may be used regardless of thesize of the current block. In this case, according to the size of thecurrent block, the number of sub-blocks included in the current blockmay be different.

Alternatively, information representing a size of a sub-block may besignaled in a bitstream. The information may be signaled in a picturelevel, a slice level or a block level.

Information for determining at least one of a position of a corner usedfor affine inter prediction or the number of motion vector parametersmay be signaled in a bitstream. In an example, information representingwhether the number of motion vector parameters is 4 or 6 may be signaledin a bitstream. When the information represents that the number ofmotion vector parameters is 4, motion compensation may be performed byusing a motion vector of a left-top corner and a motion vector of aright-top corner. When the information represents that the number ofmotion vector parameters is 6, motion compensation may be performed byusing a motion vector of a left-top corner, a motion vector of aright-top corner and a motion vector of a left-bottom corner.

Alternatively, information for specifying a position of three among fourcorners of a current block or information for specifying a position oftwo among four corners of a current block may be signaled in abitstream.

In another example, based on at least one of a size, a shape or a interprediction mode of a current block, the number of motion vectorparameters may be determined. In this case, the size of the currentblock may be represented with width, height, the sum of width and heightor the product of width and height. The shape of the current block mayrepresent at least one of whether the current block is square, whetherthe current block is non-square, whether width is greater than height ora width and height ratio. An inter prediction mode includes a merge modeand/or an AMVP mode.

In an example, according to whether a current block is square, thenumber of motion vector parameters may be determined differently. In anexample, when the current block is square (e.g., N×N), affine predictionmay be performed by using three corner motion vectors. When the currentblock is non-square (e.g., N×2N or 2N×N), affine prediction may beperformed by using two corner motion vectors.

Alternatively, according to a result from comparing the size of thecurrent block with a threshold value, the number of motion vectorparameters may be determined. In an example, when the size of thecurrent block is greater than the threshold value, affine prediction maybe performed by using three corner motion vectors. When the size of thecurrent block is smaller than the threshold value, affine prediction maybe performed by using two corner motion vectors. The threshold value maybe a predefined value in an encoder/decoder. Alternatively, informationrepresenting the threshold value may be signaled in a bitstream. Theinformation may be signaled in a sequence level, a picture level, aslice level or a block level.

Alternatively, a motion vector of a left-top corner may be used as adefault and whether a motion vector of a right-top corner is used and/orwhether a motion vector of a left-bottom corner is used may bedetermined based on a width and/or height of a current block. In anexample, based on whether the width of the current block is equal to orgreater than a threshold value, whether a motion vector of a right-topcorner is used may be determined. In addition, based on whether thewidth of the current block is equal to or greater than a thresholdvalue, whether a motion vector of a left-bottom corner is used may bedetermined.

In another example, based on at least one of a size, a shape or an interprediction mode of a current block, a position of a corner may bedetermined.

FIG. 28 is a diagram illustrating an example in which a position of acorner is determined according to a shape of a current block.

When a current block has a shape (N×2N) that height is longer thanwidth, affine prediction may be performed by using a motion vector of aleft-top corner and a motion vector of a left-bottom corner. (refer toFIG. 28 (a))

When a current block has a shape (2N×N) that width is longer thanheight, affine prediction may be performed by using a motion vector of aleft-top corner and a motion vector of a right-top corner. (refer toFIG. 28 (b))

In the after-mentioned embodiments, a corner motion vector is referredto as an affine vector. In addition, each affine vector at a pluralityof corners may be identified by prefixing an ordinal number to an affinevector. In an example, a first affine vector represents an affine vectorof a first corner, a second affine vector represents an affine vector ofa second corner and a third affine vector represents an affine vector ofa third corner. In addition, the motion vector of each sub-block derivedbased on affine vectors is referred to as an affine motion vector.

When motion compensation for a current block (e.g., a coding block or aprediction block) is performed based on an affine motion vector, it maybe defined as an affine inter mode.

Based on a size and/or shape of a current block, whether an affine intermode is allowed may be determined. In an example, when the size of thecurrent block is smaller than a threshold value, the affine inter modemay not be allowed. Alternatively, when the current block is non-square,the affine inter mode may not be allowed. In a detailed example, onlywhen the current block is square and its size is equal to or greaterthan 16×16, the affine inter mode may be allowed.

Information indicating whether an affine inter mode is applied to acurrent block may be signaled in a bitstream. The information may besignaled when an encoding mode of the current block is determined as aninter mode. When the affine inter mode is not allowed for the currentblock, encoding of the information may be omitted. The information maybe a 1-bit flag. In an example, when the flag is 1, it represents thatthe affine inter mode is applied and when the flag is 0, it representsthat the affine inter mode is not applied. Alternatively, theinformation may be an index indicating any one of a plurality of motionmodels.

Hereinafter, a motion compensation process based on an affine inter modewill be described in detail.

FIG. 29 is a flowchart illustrating a motion compensation process underan affine inter mode.

At least one of a position of a corner for an affine inter mode or thenumber of corner motion vector parameters may be determined S2910.Information for determining at least one of the position of the corneror the number of corner motion vectors may be signaled in a bitstream.Alternatively, based on at least one of a size, a shape or an interprediction mode of a current block, at least one of the position of thecorner or the number of corner motion vectors may be determined.

Based on a motion vector of a neighboring block, an affine vector ateach corner may be derived S2920. Concretely, an affine vector at eachcorner may be derived from an affine vector of the neighboring block ora translation motion vector of the neighboring block.

FIG. 30 and FIG. 31 are a diagram illustrating an example of deriving anaffine vector of a current block based on a motion vector of aneighboring block.

FIG. 30 shows a 4-parameter mode and FIG. 31 shows a 6-parameter mode.FIG. 30 (a) represents a 4-parameter mode in which an affine vector of aleft-top corner and an affine vector of a right-top corner are used andFIG. 30 (b) represents a 4-parameter mode in which an affine vector of aleft-top corner and an affine vector of a left-bottom corner are used.

An affine vector of a corner may be obtained based on a motion vector ofa neighboring block adjacent to the corner. In this case, theneighboring block may include at least one of a neighboring blockadjacent to a top of the corner, a neighboring block adjacent to a leftof the corner or a neighboring block adjacent to a diagonal direction ofthe corner.

In an example shown in FIG. 30 (a), FIG. 30 (b) and FIG. 31, an affinevector of a left-top corner may be derived based on at least one of aneighboring block A, B or C adjacent to the left-top corner. An affinevector of a right-top corner may be derived based on at least one of aneighboring block D or E adjacent to the right-top corner. An affinevector of a left-bottom corner may be derived based on at least one of aneighboring block F or G adjacent to the left-bottom corner.

After searching neighboring blocks in a predefined order, an affinevector of a corner may be derived based on a motion vector of aneighboring block which is firstly determined as available. In anexample shown in FIG. 30 (a), FIG. 30 (b) and FIG. 31, an affine vectorof a left-top corner may be derived based on a motion vector of aneighboring block which is firstly determined as available whenneighboring blocks A, B and C adjacent to the left-top corner aresequentially searched. An affine vector of a right-top corner may bederived based on a motion vector of a neighboring block which is firstlydetermined as available when neighboring blocks D and E adjacent to theright-top corner are sequentially searched. An affine vector of aleft-bottom corner may be derived based on a motion vector of aneighboring block which is firstly determined as available whenneighboring blocks F and G adjacent to the left-bottom corner aresequentially searched.

Alternatively, information for specifying any one of neighboring blocksneighboring each corner may be signaled in a bitstream. In an example,information for identifying one of neighboring blocks adjacent to aleft-top corner, information for identifying one of neighboring blocksadjacent to a right-top corner or information for identifying one ofneighboring blocks adjacent to a left-bottom corner may be signaled in abitstream. An affine vector of a corner may be determined based on amotion vector of a neighboring block which is specified by theinformation.

An affine vector may be derived from a non-adjacent block which is notadjacent to a corner. The non-adjacent block may include at least one ofa block on the same line as a block adjacent to the corner or a blockwhich is not adjacent to the corner among blocks adjacent to a currentblock. In an example, in an example shown in FIG. 30 and FIG. 31, anaffine vector of a left-top corner may be derived based on a blockexcluding neighboring blocks A, B and C (e.g., a neighboring block D, E,F or G) or a block on the same line as a neighboring block A, B or C.

An affine vector of a corner may be set the same as a motion vector of aneighboring block. Alternatively, the motion vector of the neighboringblock may be set as a motion vector prediction value and the affinevector of the corner may be derived by adding a motion vector differencevalue to the motion vector prediction value. The motion vectordifference value representing a difference between the affine vector andthe motion vector prediction value may be signaled in a bitstream.

In another example, based on a difference coding of a first affinevector and a second affine vector, the second affine vector may bederived. For it, a difference vector representing a difference betweenthe second affine vector and the first affine vector may be encoded in abitstream. A decoder may derive the second affine vector by adding adifference vector decoded in a bitstream to the first affine vector.When affine vectors of three corners are used, a third affine vector maybe derived based on a difference coding between the third affine vectorand the first affine vector or a difference coding between the thirdaffine vector and the second affine vector.

In another example, when a value of an affine vector of a first corneris the same as that of an affine vector of a second corner, or when aneighboring block used to derive the affine vector of the first cornerand the affine vector of the second corner is the same, the affinevector of the second corner may be derived from a block which is notadjacent to the second corner. In an example, in an example shown inFIG. 30 (a), when an affine vector of a left-top corner and an affinevector of a right-top corner are the same, an affine vector of aright-top corner may be derived based on the motion vector of a block For G which is not adjacent to the right-top corner.

Alternatively, when a value of an affine vector of a first corner is thesame as that of an affine vector of a second corner, or when aneighboring block used to derive the affine vector of the first cornerand a neighboring block used to derive the affine vector of the secondcorner are the same, the affine vector of the second corner may bederived based on the affine vector of the first corner. Concretely, thesecond affine vector may be derived by adding or subtracting an offsetto or from the first affine vector or by scaling the first affine vectorbased on a predefined scaling factor.

Information indicating whether affine vectors are the same may besignaled in a bitstream. The information may include at least one of aflag indicating whether there exist affine vectors identical to eachother or an index for identifying a position of corners whose affinevectors the same each other.

According to a shape of a current block, the number and/or position of acandidate neighboring block used to derive an affine vector may bedifferent. In an example, according to a shape of a prediction unit(whether it is a square shape or a non-square shape), candidate blocksfor deriving a corner affine vector may be constructed differently. Inan example, when a current block is square as in an example shown inFIG. 30 (a), an affine vector of a left-top corner may be derived basedon any one of a neighboring block A, B or C. On the other hand, when acurrent block is non-square as in an example shown in FIG. 30 (b), anaffine vector of a left-top corner may be derived based on any one of aneighboring block A, B or F.

According to an embodiment of the present disclosure, an affine mergemode for deriving affine vectors of a current block may be defined. Theaffine merge mode represents a method of deriving motion information ofa merge candidate as that of a current block. In this case, motioninformation may include at least one of a motion vector, a referencepicture index, a reference picture list or a prediction direction. Amerge candidate may include at least one of a spatial merge candidate ora temporal merge candidate. The spatial merge candidate represents amerge candidate derived from a block included in the same picture as thecurrent block and the temporal merge candidate represents a mergecandidate derived from a block included a different picture from thecurrent block.

FIG. 32 is a diagram illustrating candidate blocks for deriving aspatial merge candidate.

A spatial merge candidate may be derived from at least one of aneighboring block adjacent to a current block or a block on the sameline as the neighboring block. A block on the same line as theneighboring block may include at least one of a block on the samehorizontal line, a block on the same vertical line or a block on thesame diagonal line as the neighboring block adjacent to the currentblock.

A merge candidate derived from a block having affine motion informationmay be referred to as an inherited merge candidate. The inherited mergecandidate may be derived from a spatial neighboring block or temporalneighboring block of the current block and the derived inherited mergecandidate may be added to a merge candidate list. Alternatively, only aninherited merge candidate which is firstly searched in a predeterminedorder among left neighboring blocks of the current block, and only aninherited merge candidate which is firstly searched in a predeterminedorder among top neighboring blocks of the current block may be added toa merge candidate list. In this case, left neighboring blocks mayinclude A0 and A3 and top neighboring blocks may include A1, A2 and A4.

Motion information of an inherited merge candidate may be obtained basedon motion information of a candidate block. In an example, a referencepicture index and prediction direction of an inherited merge candidatemay be set the same as a candidate block. Affine vectors of theinherited merge candidate may be derived based on affine vectors of thecandidate block. In this case, the affine vectors of the inherited mergecandidate may include at least one of an affine vector of a left-topcorner, an affine vector of a right-top corner or an affine vector of aleft-bottom corner according to the number of parameters. In this case,corner affine vectors of the candidate block used to derive the affinevectors of the inherited merge candidate may be different according towhether a candidate block adjoins a boundary of CTU. In an example, whenthe current block and the candidate block belong to the same CTU, theaffine vectors of the merge candidate may be derived based on an affinevector of a left-top corner of the candidate block and an affine vectorof a right-top corner of the candidate block. On the other hand, whenthe current block and the candidate block belong to a different CTU, theaffine vectors of the merge candidate may be derived based on an affinevector of the left-bottom corner of the candidate block and an affinevector of the right-bottom corner of the candidate block.

Next, a constructed merge candidate may be added to a merge candidatelist. The constructed merge candidate may be generated by combiningtranslation motion information of a plurality of blocks. In an example,the constructed merge candidate may be generated by combining two ormore of candidate blocks shown in FIG. 32. When the constructed mergecandidate is constructed, candidate blocks may be combined so that anaffine vector of a corner is derived based on a translation motionvector of a neighboring block adjacent to the corner. In an example, incase of a 6-parameter mode, the constructed merge candidate may begenerated by combining a block adjacent to a left-top corner of thecurrent block, a block adjacent to a right-top corner of the currentblock and a block adjacent to a left-bottom corner of the current block.

A combination priority for generating the constructed merge candidatemay be predefined in an encoder and a decoder. The constructed mergecandidate may be generated by combining a plurality of blocks accordingto the combination priority. In this case, a combination of blocks thata reference picture is not identical may be determined to be unavailableas the constructed merge candidate.

When a constructed merge candidate is selected, a translation motionvector of each block constructing the constructed merge candidate may beset as an affine vector of a corner adjacent thereto.

On the other hand, an affine merge candidate may be derived by using oneor more temporal neighboring blocks which belong to a picture differentfrom a current picture. Concretely, when a temporal neighboring block isencoded by affine inter prediction, an inherited merge candidate may bederived from the temporal neighboring block.

Based on neighboring blocks adjacent to a current block, a first mergecandidate list may be constructed and when the number of mergecandidates included in the first merge candidate list is smaller thanthe maximum number, a merge candidate included in a second mergecandidate list may be added to the first merge candidate list.Concretely, a merge candidate not included in the first merge candidatelist among merge candidates included in the second merge candidate listmay be added to the first merge candidate list.

In this case, the first merge candidate list may include an inheritedmerge candidate and/or a constructed merge candidate derived fromneighboring blocks adjacent to a current block. The second mergecandidate list may include an inherited merge candidate and/or aconstructed merge candidate derived from blocks which are not adjacentto the current block. Alternatively, the second merge candidate list mayinclude a merge candidate derived based on affine motion information ofa block encoded/decoded by affine inter prediction before the currentblock. Constructing the second merge candidate list and adding a mergecandidate included in the second merge candidate list to the first mergecandidate list may follow the above-described embodiment.

An encoder selects a merge candidate with the highest coding efficiencyamong merge candidates included in a merge candidate list (e.g., aspatial merge candidate and a temporal merge candidate) and signalsinformation specifying the selected merge candidate. A decoder mayselect a merge candidate based on information signaled in a bitstreamand obtain motion information of a current block based on motioninformation of the selected merge candidate. Concretely, the motioninformation of the current block may be set the same as that of themerge candidate.

FIG. 33 shows an example of deriving the affine vectors of a currentblock from the affine vectors of a candidate block.

As described above, affine vectors of a merge candidate may be derivedbased on affine vectors of a block encoded by affine inter prediction.Accordingly, selecting the merge candidate may be consequentiallyconsidered to derive affine vectors of a current block from affinevectors of a candidate block.

In an example shown in FIG. 33, when A4 is used in an affine merge mode,it may be understood that at least one of affine vectors, v′0, v′1 orv′2, of a current block, is derived based on at least one of affinevectors, v0, v1 or v2, of a block corresponding to A4.

Alternatively, it may be understood that at least one of affine vectors,v′0, v′1 or v′2, of a current block is derived based on at least one ofaffine vectors, v0, v1 or v2, of a partial region corresponding to A4.In this case, the partial region may be any one of a slice, a tile, CTB,a CTB column, CU or PU including the position of A4 (e.g., (−1, −1)).

According to the size and/or shape of a current block, whether an affinemerge mode is allowed may be determined. In an example, the affine mergemode may be allowed only when the size of the current block is greaterthan a threshold value. The threshold value may be a fixed value whichis preset in an encoder and a decoder. Alternatively, information fordetermining the threshold value may be signaled in a bitstream.

Whether an affine merge mode is used for a current block may bedetermined based on a flag indicating whether an affine merge mode isused. The flag may be derived based on at least one of a size, a shape,a partitioning depth, a partition type or a component type of a currentblock. Alternatively, the flag may be encoded and signaled in abitstream.

When the flag indicates that an affine merge mode is not used, affinevectors of a current block may be derived by adding an affine vectordifference value to an affine vector prediction value.

An affine vector prediction value may be set the same as an affinevector of a motion vector candidate. The motion vector candidate may bederived from at least one of a neighboring block or a non-adjacent blockencoded by affine inter prediction. In an example, a motion vectorcandidate may be derived from a block encoded by affine inter predictionwhich is firstly searched when left neighboring blocks of the currentblock are searched in the predetermined order. In addition, a motionvector candidate may be derived from a block encoded by affine interprediction which is firstly searched when top neighboring blocks of thecurrent block are searched in the predetermined order. In this case,left neighboring blocks may include A0 and A3 and top neighboring blocksmay include A1, A2 and A4.

Affine vectors of a motion vector candidate may be derived based onaffine vectors of a candidate block. In this case, the affine vectors ofthe motion vector candidate may include at least one of an affine vectorof a left-top corner, an affine vector of a right-top corner or anaffine vector of a left-bottom corner according to the number ofparameters. The corner affine vectors of the candidate block used toderive the affine vectors of the motion vector candidate may bedifferently determined according to whether the candidate block adjoinsa boundary of CTU. In an example, when the current block and thecandidate block belong to the same CTU, affine vectors of a motionvector candidate may be derived based on an affine vector of a left-topcorner of the candidate block and an affine vector of a right-top cornerof the candidate block. On the other hand, when the current block andthe candidate block belong to a different CTU, affine vectors of themotion vector candidate may be derived based on an affine vector of aleft-bottom corner of the candidate block and an affine vector of aright-bottom corner of the candidate block.

Information for specifying any one of a plurality of motion vectorcandidates may be signaled in a bitstream. When a motion vectorcandidate is selected by the information, affine vectors of the selectedmotion vector candidate may be set as an affine vector prediction value.Affine vector of a current block may be derived by adding an affinevector difference value decoded in a bitstream to the affine vectorprediction value.

When affine vectors of a current block are derived, affine motionvectors of sub-blocks may be derived based on the affine vectors 52930.Based on an obtained affine motion vector, motion compensation persub-block may be performed 52940.

The above-mentioned example described that an affine merge mode isseparate from a general merge mode. According to an embodiment of thepresent disclosure, the general merge mode and the affine merge mode maybe used by unifying them. In an example, a merge candidate may bederived from at least one of a neighboring block adjacent to a currentblock or a non-adjacent block and a combined merge candidate listincluding the merge candidate may be generated. In this case, when acandidate block is encoded as a translation motion model, a mergecandidate derived from the candidate block may include translationmotion information. In addition, when a candidate block is encoded as anaffine motion model, a merge candidate derived from the candidate blockmay include affine motion information.

According to a merge candidate selected from the merge candidate list, amotion model of a current block may be determined. In an example, when aselected merge candidate is derived from a block encoded/decoded by anaffine mode, affine inter prediction may be performed for the currentblock.

Alternatively, when a prediction mode of a neighboring blockcorresponding to a selected merge candidate is different from that of acurrent block, motion information of the current block may be derived byusing at least only part of motion information of the selected mergecandidate. In this case, the prediction mode may include at least one ofan affine merge mode, an affine inter mode, a merge mode, an AMVP modeor a mode which is predefined in an encoder and a decoder.

In an example, when a neighboring block corresponding to a selectedmerge candidate is encoded by an affine inter mode and a current blockis encoded by a general merge mode, a motion vector of the current blockmay be derived by using all (e.g., v0, v1 and v2) or part (e.g., v0 andv1) of affine vectors of the neighboring block.

Alternatively, when a neighboring block corresponding to the selectedmerge candidate is encoded by a first prediction mode and the currentblock is encoded by a second prediction mode, a motion vector among themotion information of the merge candidate may be referred to, but areference picture index may not be referred to.

The application of the embodiments described focusing on the decodeprocess or encoding process to the encoding process or decoding processis included in the scope of the present invention. The change of theembodiments described in a predetermined order into a different order isalso included in the scope of the present invention.

Although the above-described embodiments have been described on thebasis of a series of steps or flowcharts, they are not intended to limitthe inventive time-series order, and may be performed simultaneously orin a different order. In addition, each of the components (for example,units, modules, etc.) constituting the block diagram in theabove-described embodiment may be implemented as a hardware device orsoftware, and a plurality of components may be combined into onehardware device or software. The above-described embodiments may beimplemented in the form of program instructions that may be executedthrough various computer components and recorded in a computer-readablerecording medium. The computer-readable storage medium may include aprogram instruction, a data file, a data structure, and the like eitheralone or in combination thereof. Examples of the computer-readablestorage medium include magnetic recording media such as hard disks,floppy disks and magnetic tapes; optical data storage media such asCD-ROMs or DVD-ROMs; magneto-optical media such as floptical disks; andhardware devices, such as read-only memory (ROM), random-access memory(RAM), and flash memory, which are particularly structured to store andimplement the program instruction. The hardware devices may beconfigured to be operated by one or more software modules or vice versato conduct the processes according to the present invention.

INDUSTRIAL APPLICABILITY

The present invention may be applied to an electronic device capable ofencoding/decoding an image.

1-15. (canceled)
 16. A method of decoding a video, the methodcomprising: deriving an affine merge candidate from a neighboring blockadjacent to a current block; generating a merge candidate list includingthe affine merge candidate; specifying one of a plurality of mergecandidates included in the merge candidate list; deriving first motionvectors of the current block based on the specified merge candidate;deriving a second motion vector of the current block based on the firstmotion vectors of the current block, the second motion vector beingderived in units of sub-blocks in the current block, each of thesub-blocks having a size of 4×4; and performing motion compensation forthe current block based on the second motion vector, wherein the affinemerge candidate is derived based on corner motion vectors of theneighboring block, and wherein, based on whether the neighboring blockis included in the same CTU (Coding Tree Unit) as the current block,positions of corners corresponding to the corner motion vectors used toderive the affine merge candidate are differently determined.
 17. Themethod of claim 16, wherein when the neighboring block is included inthe same CTU as the current block, a top-left corner motion vector and atop-right corner motion vector of the neighboring block are used toderive the affine merge candidate, and wherein when the neighboringblock is not included in the same CTU as the current block, abottom-left corner motion vector and a bottom-right corner motion vectorof the neighboring block are used to derive the affine merge candidate.18. The method of claim 17, wherein the merge candidate list furtherincludes a constructed affine merge candidate which is generated bycombining translation motion vectors of a plurality of neighboringblocks.
 19. The method of claim 18, wherein the affine merge candidateis derived from one of top neighboring blocks adjacent to the currentblock, and wherein the affine merge candidate is derived from availableone which is found firstly by searching the top neighboring blocks in apre-defined order.
 20. A method of encoding a video, the methodcomprising: encoding index information which specifies one of aplurality of merge candidates included in a merge candidate list; andgenerating a bitstream including the index information, wherein firstmotion vectors of the current block are determined based on the one ofthe plurality of merge candidates included in the merge candidate list,wherein a second motion vector of the current block is derived, in unitsof sub-blocks in the current block, based on the first motion vectors,wherein each of the sub-blocks has a size of 4×4, wherein the mergecandidate list includes an affine merge candidate which is derived basedon corner motion vectors of a neighboring block adjacent to a currentblock, and wherein, based on whether the neighboring block is includedin the same CTU (Coding Tree Unit) as the current block, positions ofcorners corresponding to the corner motion vectors used to derive theaffine merge candidate are differently determined.
 21. The method ofclaim 20, wherein when the neighboring block is included in the same CTUas the current block, a top-left corner motion vector and a top-rightcorner motion vector of the neighboring block are used to derive theaffine merge candidate, and wherein when the neighboring block is notincluded in the same CTU as the current block, a bottom-left cornermotion vector and a bottom-right corner motion vector of the neighboringblock are used to derive the affine merge candidate.
 22. The method ofclaim 21, wherein the merge candidate list further includes aconstructed affine merge candidate which is generated by combiningtranslation motion vectors of a plurality of neighboring blocks.
 23. Themethod of claim 22, wherein the affine merge candidate is derived fromone of top neighboring blocks adjacent to the current block, and whereinthe affine merge candidate is derived from available one which is foundfirstly by searching the top neighboring blocks in a pre-defined order.24. A non-transitory computer-readable storage medium storing abitstream comprising: index information which specifies one of aplurality of merge candidates included in a merge candidate list,wherein first motion vectors of the current block are derived based on amerge candidate specified by the index information, wherein a secondmotion vector of the current block is derived, in units of sub-blocks inthe current block, based on the first motion vectors, wherein each ofthe sub-blocks has a size of 4×4, wherein the merge candidate listincludes an affine merge candidate which is derived based on cornermotion vectors of a neighboring block adjacent to a current block, andwherein, based on whether the neighboring block is included in the sameCTU (Coding Tree Unit) as the current block, positions of cornerscorresponding to the corner motion vectors used to derive the affinemerge candidate are differently determined.